Wearable electronic device comprising infrared sensing camera

Abstract

In an embodiment, the wearable electronic device may comprise: a housing configured to be wearable on the head of a user; a display panel positioned in the housing to generate visible light toward at least a first direction; a lens assembly positioned in the first direction of the display panel; at least one infrared light source positioned on the display panel to radiate infrared light toward the eyeballs of the user; and a camera positioned in a second direction opposite to the first direction of the display panel to sense infrared light reflected from the eyeballs of the user. The camera may be positioned outside the effective field of view of the display panel.

Description

Wearable electronic device including an infrared sensing camera

[0001] Various embodiments of this document relate to a wearable electronic device including an infrared sensing camera.

[0002] A variety of wearable electronic devices, such as virtual reality (VR) devices, augmented reality (AR) devices, and / or mixed reality (MR) devices, are becoming commercially available. Wearable electronic devices can be configured to fit on a user's head. For example, wearable electronic devices can take the form of glasses or goggles.

[0003] In one embodiment, a wearable electronic device may include a housing configured to be worn on a user's head, a display panel positioned in the housing to generate visible light in at least a first direction, a lens assembly positioned on a side of the display panel in the first direction, at least one infrared light source positioned on the display panel to irradiate infrared light toward an eye of the user, and a camera positioned on a side of the display panel in a second direction opposite to the first direction to sense infrared light reflected from the eye of the user. The camera may be positioned outside an effective field of view of the display panel.

[0004] In one embodiment, a display assembly configured to be applied to a wearable electronic device may include a display panel that generates visible light in at least a first direction, a lens assembly positioned on a side of the display panel in the first direction, at least one infrared light source positioned on the display panel that irradiates infrared light toward a user's eye, and a camera positioned on a side of the display panel in a second direction opposite to the first direction that senses infrared light reflected from the user's eye. The camera may be positioned outside an effective viewing area of ​​the display panel.

[0005] In one embodiment, a wearable electronic device may include a housing configured to be worn on a user's head, a display panel positioned in the housing to generate visible light in at least a first direction, a lens assembly positioned on a side of the display panel in the first direction, a polarizing plate positioned on the side of the lens assembly in the first direction, at least one infrared light source positioned on the display panel to irradiate infrared light toward an eye of the user, and a camera positioned on a second direction side of the display panel opposite to the first direction to sense infrared light reflected from the eye of the user. The camera may be positioned on the second direction side of the display panel outside an effective field of view of the display panel using an under display camera (UDC) structure. The lens assembly may be configured to provide a first infrared path through which infrared light irradiated from the infrared light source reaches the eye of the user, and a second infrared path through which infrared light reflected from the eye of the user reaches the camera. The polarizing plate may form the first infrared path or the second infrared path.

[0006] In one embodiment, the performance of the camera can be improved and user convenience can be enhanced by positioning the camera on the back of the display panel so that the camera is positioned in the direction of the user's gaze.

[0007] In one embodiment, by positioning an infrared light source on a display panel, space utilization and aesthetics can be improved, and human hazards can be reduced.

[0008] The effects of the wearable assembly according to various embodiments are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

[0009] FIG. 1 is a block diagram of an electronic device within a network environment according to one embodiment.

[0010] FIG. 2A is a drawing showing the front side of a wearable electronic device according to one embodiment.

[0011] FIG. 2b is a drawing showing the back of a wearable electronic device according to one embodiment.

[0012] FIG. 3A is a schematic cross-sectional view of a display assembly according to one embodiment, illustrating a visible light path.

[0013] FIG. 3b is a schematic cross-sectional view of a display assembly according to one embodiment, showing an infrared path.

[0014] FIG. 4a is a front view showing an infrared light source arranged on a display panel according to one embodiment.

[0015] FIG. 4b is a front view showing a state in which an infrared light source is arranged on a display panel according to one embodiment.

[0016] FIG. 5A is a front view showing a state in which a camera is arranged on a display panel according to one embodiment.

[0017] FIG. 5b is a front view showing a state in which a camera is arranged on a display panel according to one embodiment.

[0018] FIG. 6 is a cross-sectional view of a display panel and a camera according to one embodiment.

[0019] FIG. 7 is a schematic cross-sectional view of a display assembly according to one embodiment, showing an infrared path.

[0020] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are assigned the same reference numerals regardless of the drawing numbers, and redundant descriptions thereof will be omitted.

[0021]

[0022] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to an embodiment. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with the electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of the electronic device (104) or the server (108) via a second network (199) (e.g., a long-range wireless communication network). According to an embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In one embodiment, the electronic device (101) may have at least one of these components (e.g., the connection terminal (178)) omitted, or one or more other components added. In one embodiment, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0023] The processor (120) may control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) by executing, for example, software (e.g., a program (140)), and may perform various data processing or calculations. According to one embodiment, as at least a part of the data processing or calculation, the processor (120) may store a command or data received from another component (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the command or data stored in the volatile memory (132), and store the resulting data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or a secondary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together therewith. For example, if the electronic device (101) includes a main processor (121) and a secondary processor (123), the secondary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a specified function. The secondary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0024] The auxiliary processor (123) may control at least a part of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0025] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0026] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0027] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0028] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. According to one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0029] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0030] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0031] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0032] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0033] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., the electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0034] A haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0035] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0036] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0037] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0038] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0039] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) may support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0040] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the selected at least one antenna. According to one embodiment, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0041]

[0042] In one embodiment, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0043] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0044] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0045]

[0046] Electronic devices according to embodiments disclosed herein may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to embodiments disclosed herein are not limited to the aforementioned devices.

[0047] The embodiments of this document and the terminology used herein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0048] The term "module" used in one embodiment of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0049] An embodiment of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0050] According to one embodiment, the method according to one embodiment disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0051] According to one embodiment, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to one embodiment, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to one embodiment, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0052]

[0053] FIGS. 2A and 2B are diagrams illustrating the front and back of a wearable electronic device (200) according to one embodiment. When a user wears the wearable electronic device (200), the appearance seen by the user's eyes may be as shown in FIG. 2B.

[0054] Referring to FIG. 2A, according to various embodiments, the electronic device (101) of FIG. 1 may include a wearable electronic device (200) that provides a service that provides an extended reality (XR) experience to a user. For example, XR or XR service may be defined as a service that collectively refers to virtual reality (VR), augmented reality (AR), and / or mixed reality (MR).

[0055] According to one embodiment, the wearable electronic device (200) may have a form factor for being worn on a user's head. The wearable electronic device (200) may refer to a head-mounted device or a head-mounted display worn on the user's head, but may also be configured in the form of at least one of glasses, goggles, a helmet, or a hat. The wearable electronic device (200) may include an OST (optical see-through) type configured to allow external light to reach the user's eyes through glasses when worn, or a VST (video see-through) type configured to allow light emitted from a display to reach the user's eyes when worn, but block external light so that external light does not reach the user's eyes.

[0056] According to one embodiment, the wearable electronic device (200) may be worn on the user's head and provide the user with an image related to an extended reality (XR) service. For example, the wearable electronic device (200) may provide XR content (hereinafter referred to as an XR content image) that outputs at least one virtual object to be superimposed on a display area or an area determined to be the user's field of view (FoV). According to one embodiment, the XR content may refer to an image or image related to a real space acquired through a camera (e.g., a camera for taking pictures) or an image or image in which at least one virtual object is superimposed on a virtual space. According to one embodiment, the wearable electronic device (200) may provide XR content based on a function being performed by the wearable electronic device (200) and / or a function being performed by one or more external electronic devices (e.g., the electronic devices 102, 104, or 108 of FIG. 1).

[0057] According to one embodiment, the wearable electronic device (200) is at least partially controlled by an external electronic device (e.g., electronic devices (102 or 104) of FIG. 1), and may perform at least one function under the control of the external electronic device, but may also perform at least one function independently.

[0058] Referring to FIGS. 2A and 2B , a wearable electronic device (200) may include a housing (210) in which at least some of the components of FIG. 1 are arranged. The housing (210) may be configured to be wearable on a user's head. For example, the housing (210) may include a strap and / or a wearing member for being fixed on a body part of the user. For example, the user may wear the wearable electronic device (200) on the head so that the first direction (①) of the wearable electronic device (200) is faced.

[0059] Referring to FIG. 2B, a fourth function camera (e.g., a face recognition camera) (225, 226, 227) and / or a display assembly (300) may be disposed in a first direction (①) of the housing (210) facing the user's face. Referring to FIG. 2A, a first function camera (e.g., a recognition camera) (215), a second function camera (e.g., a shooting camera) (211, 212), a depth sensor (217), and / or a touch sensor (213) may be disposed in a second direction (②) of the housing (210) opposite to the first direction (①). Although not illustrated in the drawing, the housing (210) may include a memory (e.g., a memory (130) of FIG. 1) and a processor (e.g., a processor (120) of FIG. 1), and may further include other components illustrated in FIG. 1.

[0060] In one embodiment, the display assembly (300) may be positioned in the first direction (①) of the wearable electronic device (200). For example, the display assembly (300) may be positioned toward the user's face. The display assembly (300) may include a display panel (e.g., the display module (160) of FIG. 1 and / or the display panel (310) of FIG. 3A) and a lens assembly (e.g., the lens assembly (320) of FIG. 3A).

[0061] According to one embodiment, the display assembly (300) may include a liquid crystal display (LCD), a digital mirror device (DMD), a liquid crystal on silicon (LCoS), an organic light emitting diode (OLED), or a micro light emitting diode (micro LED).

[0062] In one embodiment, when the display assembly (300) is formed of one of a liquid crystal display (LCD), a digital mirror display (DMD), or a silicon liquid crystal display (SiLCD), the wearable electronic device (200) may include a light source that irradiates light (e.g., visible light) to a screen output area of ​​the display assembly (300). In another embodiment, when the display assembly (300) can generate light (e.g., visible light) on its own, for example, when the wearable electronic device (200) is formed of one of an organic light-emitting diode (OLED) or a micro LED, the wearable electronic device (200) may provide a user with good quality XR content images even without including a separate light source. For example, if the display assembly (300) is implemented with an organic light-emitting diode (OLED) or a micro LED, a light source is unnecessary, and thus the electronic wearable electronic device (200) may be lightweight.

[0063] According to one embodiment, the display assembly (300) may include a first display assembly (300a) and / or a second display assembly (300b). According to one embodiment, the first display assembly (300a) may be arranged to face the user's left eye in the fourth direction (④), and the second display assembly (300b) may be arranged to face the user's right eye in the third direction (③).

[0064] According to one embodiment, the display assembly (300) may include a lens assembly (e.g., lens assembly (320) of FIG. 3A) including a transparent waveguide. The lens assembly (320) may serve to adjust a focus so that a screen (e.g., XR content image) output from a display panel (e.g., display panel (310) of FIG. 3A) can be viewed by a user. For example, light (e.g., visible light) emitted from the display panel (310) may pass through the lens assembly (320) and be transmitted to the user through a waveguide formed within the lens assembly (320). The lens assembly (320) may include at least one of a Fresnel lens, a pancake lens, a convex lens, or a multi-channel lens.

[0065] In one embodiment, the first function cameras (e.g., recognition cameras) (215) can acquire images while the wearable electronic device (200) is worn by the user. The first function cameras (215) can be used for the purpose of detecting user movements or recognizing user gestures. For example, the first function cameras (215) can be used for at least one of hand detection, hand tracking, recognition of user gestures (e.g., hand movements), and / or space recognition. For example, the first function cameras (215) mainly use GS (global shutter) cameras, which have superior performance compared to RS (rolling shutter) cameras, to detect and track fine movements of hand movements and fingers, and can be configured as a stereo camera including two or more GS cameras for head tracking and space recognition. The first function cameras (215) can be used for 3DoF, 6DoF head tracking, location (spatial, environmental) recognition, and / or movement recognition. The first function camera (215) can perform simultaneous localization and mapping (SLAM) function to recognize information (e.g., location and / or direction) related to the surrounding space through spatial recognition for 6DoF and depth shooting. In one embodiment, the second function cameras (211, 212) can also be used for hand detection and tracking, and user gestures.

[0066] In one embodiment, the second function camera (e.g., a camera for shooting) (211, 212) can obtain an image related to the surrounding environment of the wearable electronic device (200). The second function camera (211, 212) can be used to shoot the outside and generate an image or video corresponding to the outside and transmit it to a processor (e.g., the processor (120) of FIG. 1). The processor (120) can display the image provided from the second function camera (211, 212) on the display assembly (300). The second function camera (211, 212) may also be referred to as an HR (high resolution) or PV (photo video) camera and may include a high-resolution camera. For example, the second function camera (211, 212) may include a color camera equipped with functions for obtaining high-quality images, such as an auto focus (AF) function and an optical image stabilizer (OIS), but is not limited thereto, and the second function camera (211, 212) may also include a GS camera or an RS camera.

[0067] In one embodiment, a third function camera (e.g., a gaze tracking camera) (e.g., a camera (340) of FIG. 3B) may be positioned in the display assembly (300) (or inside the housing (210)) so that the camera lens faces the user's eyes when the user wears the wearable electronic device (200). The third function camera (340) may be used for the purpose of detecting and tracking pupils (ET) and / or recognizing the user's iris. The processor (120) may track the movements of the user's left and right eyes in the images received from the third function camera (340) to determine the gaze direction. The processor (120) may track the position of the pupil in the images so that the center of the XR content image displayed in the screen display area is positioned according to the direction in which the pupil is gazing. As an example, the third function camera (340) may be a GS camera to detect the pupil and track the movement of the pupil. The third function camera (340) can be installed for the left eye and the right eye, respectively, and cameras with the same performance and specifications can be used.

[0068] In one embodiment, the fourth functional camera (e.g., a face recognition camera) (225, 226, 227) may be used to detect and track (FT) the user's facial expression when the user wears the wearable electronic device (200). For example, the fourth functional camera (225, 226, 227) may be used to recognize the user's face, or to recognize and / or track the user's two eyes.

[0069] According to one embodiment, the depth sensor (or depth camera) (217) can be used for the purpose of checking the distance to an object (e.g., an object), such as TOF (time of flight). TOF (time of flight) is a technology that measures the distance to an object using a signal (e.g., near-infrared, ultrasound, or laser). After a signal is transmitted from a transmitter, a signal is measured at a receiver, and the distance to the object can be measured based on the flight time of the signal. For example, the depth sensor (217) can be configured to transmit a signal and receive a signal reflected from a subject. Instead of or in addition to the depth sensor (217), a first camera (245a, 245b, 245c, 245d) can check the distance to an object.

[0070] According to one embodiment, the touch sensor (213) may be arranged in the second direction (②) of the housing (210). The touch sensor (213) may be implemented as a single type or a left / right separated type depending on the shape of the housing (210), but is not limited thereto. For example, when the touch sensor (213) is implemented as a left / right separated type as illustrated in FIG. 2A, when the user wears the wearable electronic device (200), the first touch sensor (213a) may be arranged at the user's left eye position, such as in the fourth direction (④), and the second touch sensor (213b) may be arranged at the user's right eye position, such as in the third direction (③).

[0071] In one embodiment, the touch sensor (213) can recognize a touch input in at least one of, for example, a capacitive, pressure-sensitive, infrared, or ultrasonic manner. For example, the capacitive touch sensor (213) can recognize a physical touch (or contact) input or a hovering input (or proximity) of an external object. According to some embodiments, the wearable electronic device (200) may utilize a proximity sensor (not shown) to enable proximity recognition of an external object.

[0072] According to one embodiment, the touch sensor (213) has a two-dimensional surface and can transmit touch data (e.g., touch coordinates) of an external object (e.g., a user's finger) that comes into contact with the touch sensor (213) to a processor (e.g., the processor (120) of FIG. 1). The touch sensor (213) can detect a hovering input for an external object (e.g., a user's finger) that approaches within a first distance from the touch sensor (213), or detect a touch input that touches the touch sensor (213).

[0073] According to one embodiment, the touch sensor (213) may provide two-dimensional information about the contact point as “touch data” to the processor (120) when an external object touches the touch sensor (213). The touch data may be described as a “touch mode.” The touch sensor (213) may provide hovering data about the time or location of hovering around the touch sensor (213) to the processor (120) when an external object is located within a first distance from the touch sensor (or in proximity, hovering above the touch sensor). The hovering data may be described as a “hovering mode / proximity mode.”

[0074] According to one embodiment, the wearable electronic device (200) may obtain hovering data using at least one of a touch sensor (213), a proximity sensor (not shown), or / and a depth sensor (217) to generate information about a distance, location, or time point between the touch sensor (213) and an external object.

[0075] According to one embodiment, the interior of the housing (210) may include components of FIG. 1, for example, a processor (e.g., processor (120) of FIG. 1) and memory (e.g., memory (130) of FIG. 1).

[0076] In one embodiment, the memory (130) may store various instructions that may be performed by the processor (120). The instructions may include arithmetic and logical operations, data movement, or control commands such as input / output that may be recognized by the processor (120). The memory (130) may include volatile memory (e.g., volatile memory (132) of FIG. 1) and non-volatile memory (e.g., non-volatile memory (134) of FIG. 1), and may temporarily or permanently store various data.

[0077] In one embodiment, the processor (120) may be operatively, functionally, and / or electrically connected to each component of the wearable electronic device (200) and may be configured to perform operations or data processing related to control and / or communication of each component. Operations performed by the processor (120) may be stored in the memory (130) and, when executed, may be executed by instructions that cause the processor (120) to operate.

[0078] Hereinafter, the computational and data processing functions that the processor (120) can implement on the wearable electronic device (200) are not limited, but a series of operations related to the XR content service function will be described. The operations of the processor (120) described below can be performed by executing instructions stored in the memory (130).

[0079] According to one embodiment, the processor (120) may generate a virtual object based on virtual information based on image information. The processor (120) may output a virtual object related to an XR service together with background space information through the display assembly (300). For example, the processor (120) may capture an image related to an actual space corresponding to the field of view of a user wearing the wearable electronic device (200) through the second function camera (211, 212) to obtain image information or generate a virtual space for a virtual environment. For example, the processor (120) may control the display assembly (300) to display XR content (hereinafter referred to as an XR content screen) in which at least one virtual object is output to be overlapped in an area determined to be a display area or a field of view (FoV) of the user.

[0080]

[0081] FIG. 3A is a schematic cross-sectional view of a display assembly according to one embodiment, illustrating a visible light path. FIG. 3B is a schematic cross-sectional view of a display assembly according to one embodiment, illustrating an infrared path. FIG. 4A is a front view illustrating a state in which an infrared light source is arranged on a display panel according to one embodiment. FIG. 4B is a front view illustrating a state in which an infrared light source is arranged on a display panel according to one embodiment. FIG. 5A is a front view illustrating a state in which a camera is arranged on a display panel according to one embodiment. FIG. 5B is a front view illustrating a state in which a camera is arranged on a display panel according to one embodiment.

[0082] Referring to FIGS. 3A to 5B, a display assembly (300) according to one embodiment may be applied to a wearable electronic device (e.g., the wearable electronic device (200) of FIG. 2B). In FIGS. 3A to 5B, a first direction (①) may be a direction toward a user's face (e.g., an eye (E)), and a second direction (②) may be a direction opposite to the first direction (①).

[0083] In one embodiment, the display assembly (300) may include a display panel (310), a lens assembly (320), an infrared light source (330), a camera (340), and / or a polarizing plate (not shown).

[0084] In one embodiment, the display panel (310) can generate light (e.g., visible light) at least in a first direction (①). The display panel (310) can be positioned inside a housing (e.g., the housing (210) of FIG. 2B) of a wearable electronic device (e.g., the wearable electronic device (200) of FIG. 2B). For example, the display panel (310) can be fixedly connected to a second direction (②) side of a lens assembly (320). Visible light generated from the display panel (310) can pass through the lens assembly (320) and be transmitted to the user's eye (E). Although the display panel (310) is illustrated as having a square shape in FIGS. 4A to 5B , this is merely exemplary and the shape of the display panel (310) is not limited thereto. For example, the display panel (310) can be formed in a circular or arbitrary polygonal shape.

[0085] In one embodiment, the lens assembly (320) may be positioned on the first direction (①) side of the display panel (310). The lens assembly (320) may provide a path for visible light and / or infrared light to pass through. The lens assembly (320) may be formed to transmit both visible light and infrared light. For example, the lens assembly (320) may be formed to filter a region excluding the visible light region and the infrared region. For example, the lens assembly (320) may be configured to substantially function as a dual wavelength band-pass filter. The lens assembly (320) may include a lens barrel and / or at least one lens. For example, the lens assembly (320) may include at least one of a Fresnel lens, a pancake lens, a convex lens, or a multi-channel lens. However, this is exemplary, and the types included in the lens assembly (320) are not limited thereto. In addition, the structure and / or shape of the lens assembly (320) illustrated in FIGS. 3a and 3b are exemplary, and the structure and / or shape of the lens assembly (320) is not limited thereto.

[0086] In one embodiment, the lens assembly (320) may be configured to provide a visible light path (V) through which visible light generated from the display panel (310) reaches the user's eye (E). For example, the lens assembly (320) may magnify and / or reduce content displayed on the display panel (310) and transmit it to the user's eye (E). For example, the lens assembly (320) may adjust a focus so that content displayed on the display panel (310) is visible to the user. However, the visible light path (V) illustrated in FIG. 3A is simplified for convenience of explanation, and the visible light path (V) is not limited thereto.

[0087] In one embodiment, the infrared light source (330) can irradiate infrared rays toward the user's eyes (E). For example, the infrared light source (330) can generate infrared rays toward at least a first direction (①). At least one infrared light source (330) can be provided. The infrared light source (330) can be positioned on the display panel (310). For example, the infrared light source (330) can be positioned on the outside or inside of the display panel (310).

[0088] In one embodiment, as illustrated in FIG. 4A, at least one infrared light source (330a) may be positioned in an outer edge area (e.g., a bezel area (B)) of the display panel (310). For example, a plurality of infrared light sources (330a) may be positioned at a specified interval along the bezel area (B) of the display panel (310). With this structure, the space where the infrared light sources (330a) are positioned can be saved by utilizing the bezel area (B) of the display panel (310). In addition, since at least one infrared light source (330a) is positioned inside the housing (e.g., the housing (210) of FIG. 2B), the appearance of the wearable electronic device (e.g., the wearable electronic device (200) of FIG. 2B) can be improved. Since the distance between the infrared light source (330a) and the user's eyes (E) can be maintained at a predetermined distance, harm to the human body can be reduced.

[0089] In one embodiment, as illustrated in FIG. 4B, at least one infrared light source (330b) may be positioned inside the display panel (310). For example, at least one infrared light source (330b) may be positioned adjacent to an outer edge (e.g., a bezel area (B)) of the display panel (310). For example, a plurality of infrared light sources (330b) may be positioned at a specified interval inside the display panel (310) along the outer edge of the display panel (310). For example, at least one infrared light source (330b) may be configured as a part of a pixel of the display panel (310). For example, at least some of the pixels of the display panel (310) may be set to radiate infrared light. For example, the pixels set to radiate infrared light may be positioned outside the visible area of ​​the display panel (310). According to this structure, the space where the infrared light source (330b) is placed can be saved by utilizing the internal area of ​​the display panel (310). By positioning the infrared light source (330b) adjacent to the outer edge of the display panel (310), interference between the display area of ​​the display panel (310) and the area of ​​the infrared light source (330b) can be reduced. In addition, since at least one infrared light source (330b) is positioned inside the housing (e.g., the housing (210) of FIG. 2B), the appearance of the wearable electronic device (e.g., the wearable electronic device (200) of FIG. 2B) can be improved. Since the distance between the infrared light source (330b) and the user's eye (E) can be maintained at a certain distance, harm to the human body can be reduced.

[0090] Meanwhile, the position, size and / or number of the infrared light sources (330) illustrated in FIGS. 4A and 4B are exemplary and are not limited thereto. In one embodiment, the arrangement of at least one infrared light source (330a) and at least one infrared light source (330b) is not limited to the arrangement of the embodiment illustrated in the drawings, and may be formed in a structure in which at least one infrared light source (330a) is arranged in the bezel area (B) and at least one infrared light source (330b) is arranged inside the display panel (310). In one embodiment, at least one infrared light source (330a) and at least one infrared light source (330b) may be formed together in one display panel (310). In one embodiment, the infrared light source (330) may be positioned on the front surface (e.g., the surface in the first direction (①)) and / or the back surface (e.g., the surface in the second direction (②)) of the display panel (310) so as to be adjacent to the outer edge of the display panel (310).

[0091] In one embodiment, infrared light generated from an infrared light source (330) may pass through a lens assembly (320) and be transmitted to a user's eye (E). Infrared light reaching the user's eye (E) may be reflected by the user's eye (E), and the reflected infrared light may again pass through the lens assembly (320) and be transmitted to the camera (340). The lens assembly (320) may be configured to provide a first infrared path (R1) through which infrared light irradiated from the infrared light source (330) reaches the user's eye (E), and a second infrared path (R2) through which infrared light reflected from the user's eye (E) reaches the camera (340). For example, when the camera (340) is positioned on one side (e.g., the upper side) of the display panel (310), the second infrared path (R2) may be formed on one area (e.g., the upper area) of the lens assembly (320). However, the first infrared path (R1) and the second infrared path (R2) illustrated in FIG. 3B are simplified for convenience of explanation, and the first infrared path (R1) and the second infrared path (R2) are not limited thereto. For example, the visible light path (V) and the second infrared path (R2) may not overlap each other. For example, the second infrared path (R2) may be formed in one area (e.g., an upper area) of the lens assembly (320), and the visible light path (V) may be formed in the remaining area of ​​the lens assembly (320). For example, a partition guide may be formed in the lens assembly (320) to separate the visible light path (V) and the second infrared path (R2). In this case, one area (e.g., an upper area) of the lens assembly (320) may function as an infrared filter that passes only infrared light, and the remaining area of ​​the lens assembly (320) may function as a visible light filter that passes only visible light.

[0092] In one embodiment, the camera (340) can sense an image of the user's eye (E). For example, the camera (340) can sense infrared rays reflected from the user's eye (E). For example, the camera (340) can sense a reflected image (glint image) of the user's eye (E) and / or an iris image. For example, the camera (340) can be used to track the user's gaze and / or to recognize the user's iris. For example, a processor (e.g., the processor (120) of FIG. 1) can be configured to track the user's gaze or recognize the user's iris using the sensing information of the camera (340).

[0093] In one embodiment, the camera (340) may be positioned on the back side (e.g., the side in the second direction (②)) of the display panel (310). For example, the camera (340) may be positioned on the back side (e.g., the side in the second direction (②)) of the display panel (310) using an under display camera (UDC) structure.

[0094] In one embodiment, as illustrated in FIG. 5A, at least one camera (340a) may be positioned at a central portion (e.g., a top central portion) of one side of the display panel (310). However, this is merely exemplary, and the at least one camera (340a) may be positioned at any location of the display panel (310). For example, the at least one camera (340a) may be positioned at any one of the top, center, and bottom of the display panel (310). For example, the at least one camera (340a) may be positioned at at least a portion of the bezel area (B) of the display panel.

[0095] In one embodiment, as illustrated in FIG. 5b, at least one camera (340b) may be positioned outside the effective viewing area (F) of the display panel (310). For example, the effective viewing area (F) may refer to a display area visible to the user's field of view. For example, as illustrated in FIG. 5b, when the effective viewing area (F) is formed in a circular shape in the center of the display panel (310), at least one camera (340b) may be positioned near a corner of the display panel (310) so as to be positioned outside the effective viewing area (F) of the display panel (310). However, this is exemplary, and at least one camera (340b) may be positioned at any location outside the effective viewing area (F) of the display panel (310). When the camera (340b) is positioned outside the effective viewing area (F) of the display panel (310), the camera (340b) is not visible from the user's field of view, so that the degradation of the image quality of the display panel (310) perceived by the user can be reduced.

[0096] In one embodiment, by positioning the camera (340) on the back surface (e.g., the surface in the second direction (②)) of the display panel (310) using an under display camera (UDC) structure, the performance of the camera (340) in sensing the image of the user's eye (E) can be improved. For example, since the user wears the wearable electronic device (e.g., the wearable electronic device (200) of FIG. 2B) so that the eye (E) faces the display panel (310), the camera (340) positioned on the back surface (e.g., the surface in the second direction (②)) of the display panel (310) can be positioned in the direction of the user's gaze. Compared to the comparative example in which the camera is positioned outside the display assembly (e.g., outside the housing of the wearable electronic device), in the wearable electronic device (200) according to one embodiment, since the camera (340) is positioned in the direction of the user's gaze, the camera (340) can sense the user's eye (E) image more easily and accurately. In addition, since the camera (340) is positioned on the back surface of the display panel (310) (e.g., the surface in the second direction (②)), iris recognition can be performed while the user's gaze is directed forward. For example, in the comparative example in which the camera is positioned outside the display assembly (e.g., outside the housing of the wearable electronic device), the user may have to turn his gaze outward and gaze outside the display assembly (e.g., outside the housing of the wearable electronic device) for iris recognition. Compared to the case of these comparative examples, in the wearable electronic device (200) according to one embodiment, since the camera (340) is positioned in the direction of the user's gaze, iris recognition can be performed while the user is looking straight ahead, and thus user convenience can be improved.

[0097] In one embodiment, a wearable electronic device (e.g., the wearable electronic device (200) of FIG. 2B) may further include a sensor position correction unit (not shown). The sensor position correction unit may be configured to correct the position of an image sensor (e.g., the image sensor (342) of FIG. 6) in response to shaking of the wearable electronic device (200). For example, the sensor position correction unit may include an optical image stabilization (OIS) structure. The sensor position correction unit may be positioned together with the camera (340) or inside the camera (340). For example, the sensor position correction unit may be positioned on the back side (e.g., the surface in the second direction (②)) of the display panel (310). For example, in the comparative example where the camera is located outside the display assembly (e.g., outside the housing of the wearable electronic device), it may be difficult to install a sensor position correction unit due to the narrow space, but in the wearable electronic device (200) according to one embodiment, it may be possible to install a sensor position correction unit using the space on the back side (e.g., the surface in the second direction (②)) of the display panel (310). Accordingly, since shaking can be responded to through the sensor position correction unit, the performance of the camera (340) can be improved. Meanwhile, this is exemplary, and the structure for correcting the shaking of the wearable electronic device (200) is not limited thereto. In one embodiment, in order to correct the shaking of the wearable electronic device (200), a position correction unit configured to move a lens assembly (e.g., barrel (341) of FIG. 6) of the camera (340) may be provided.

[0098]

[0099] FIG. 6 is a cross-sectional view of a display panel and a camera according to one embodiment.

[0100] Referring to FIG. 6, in one embodiment, the camera (340) may be positioned in the camera hole area (311) of the display panel (310). The camera hole area (311) may be an area on the back surface (e.g., the surface in the second direction (②)) of the display panel (310) where the camera (340) is positioned. For example, infrared rays reflected from the eye may be incident on the camera hole area (311). For example, the camera hole area (311) may be configured to have an infrared transmittance (e.g., transmittance at a wavelength of 750 nm or more) of at least 40% or more. However, this is exemplary, and the infrared transmittance is not limited thereto.

[0101] In one embodiment, the camera (340) may include a barrel (341), an image sensor (342), a first infrared filter (343), and / or a second infrared filter (344). The camera (340) may be configured with at least 4 million pixels for iris recognition of a user. However, this is exemplary, and the pixels of the camera (340) are not limited thereto.

[0102] In one embodiment, the barrel (341) may constitute at least a portion of the housing of the camera (340). An image sensor (342) may be positioned inside the barrel (341). The image sensor (342) may convert incident infrared light into an electrical image signal. A first infrared filter (343) may be positioned on the first direction (①) side of the barrel (341). For example, the first infrared filter (343) may be attached to an end of the barrel (341) in the first direction (①). For example, the first infrared filter (343) may be positioned between the barrel (341) and the display panel (310). For example, the first infrared filter (343) may include a cover glass. A second infrared filter (344) may be positioned on the first direction (①) side of the image sensor (342). For example, the second infrared filter (344) may cover the image sensor (342) in the first direction (①). For example, the second infrared filter (344) may include a cover film. The first infrared filter (343) and the second infrared filter (344) may refer to filters configured to pass only the infrared region. With this structure, among the light incident through the display panel (310), a noise region (e.g., a visible light region) may be filtered out, and only the infrared region may be incident on the image sensor (342). Therefore, the quality of the image acquired by the image sensor (342) may be improved. Meanwhile, the structure of the first infrared filter (343) and / or the second infrared filter (344) described above is exemplary and is not limited thereto. In one embodiment, the infrared filter may be positioned as a separate structure outside the camera (340). In one embodiment, a filter that integrates the first infrared filter (343) and the second infrared filter (344) may be applied.

[0103] In one embodiment, the polarizing plate (not shown) may be configured to form at least one of a visible light path (V), a first infrared path (R1), or a second infrared path (R2). For example, the polarizing plate may form the first infrared path (R1) or the second infrared path (R2). For example, the polarizing plate may be positioned on the first direction (①) side of the lens assembly (320). For example, the polarizing plate may be positioned between the lens assembly (320) and the user's eye (E). For example, the polarizing plate may include at least one of a full-wave plate, a half-wave plate, a quarter-wave plate, a multiple-order waveplate, or a zero-order waveplate. However, this is exemplary, and the type of the polarizing plate is not limited thereto.

[0104]

[0105] Hereinafter, with reference to FIGS. 2A to 6, a process of tracking a user's gaze and / or a process of recognizing a user's iris in a wearable electronic device (200) according to one embodiment will be described.

[0106] In one embodiment, when a user wears a wearable electronic device (200), an infrared light source (330) may irradiate infrared rays toward the user's eye (E). At least a portion of the infrared rays generated from the infrared light source (330) may be transmitted to the user's eye (E) along a first infrared path (R1) provided by a lens assembly (320). The infrared rays transmitted to the user's eye (E) may be reflected to generate an infrared component eye image (e.g., a reflection image and / or an iris image). The reflected infrared rays may be transmitted to a camera (340) positioned on the back surface (e.g., the surface in the second direction (②)) of the display panel (310) along a second infrared path (R2) provided by the lens assembly (320). A processor (e.g., the processor (120) of FIG. 1) may analyze the infrared rays transmitted to the camera (340) to track the user's gaze or recognize the user's iris. For example, an image reflected from an eye (E) can be captured through a camera (340), an iris portion can be extracted from the captured image, and an iris photograph can be imaged. For example, the size, shape, color, retina, and / or capillaries of the iris can be analyzed to extract a code based on a unique iris-related pattern. The user's iris can be recognized based on this code. According to one embodiment, a gaze or a direction in which the user is looking can be extracted from the imaged iris photograph. When the wearable electronic device (200) is shaken, a sensor position correction unit can correct the position of the camera (340) to improve the accuracy of gaze tracking and / or iris recognition.

[0107]

[0108] FIG. 7 is a schematic cross-sectional view of a display assembly according to one embodiment, showing an infrared path.

[0109] Referring to FIG. 7, the display assembly (300-1) may include a display panel (310), a lens assembly (320-1), an infrared light source (e.g., the infrared light source (330) of FIG. 4A and / or the infrared light source (330) of FIG. 4B), a camera (340-1), and / or a polarizing plate (not shown). In describing the embodiment according to FIG. 7, for a configuration substantially identical to that of the embodiment described through FIGS. 3A to 6, the contents described through FIGS. 3A to 6 are applied to the extent that they do not conflict with each other.

[0110] In one embodiment, the camera (340-1) may be positioned adjacent to the display panel (310) on the outside of the display panel (310). For example, the camera (340-1) may be positioned adjacent to a side (e.g., a border) of the display panel (310). For example, the camera (340-1) may be positioned substantially on the same plane as the display panel (310) or on a plane adjacent thereto. For example, the camera (340-1) may be positioned adjacent to an upper border of the display panel (310).

[0111] In one embodiment, the camera (340-1) may be arranged at an angle with respect to the display panel (310). For example, the camera (340-1) may be arranged at an angle such that the front side (e.g., the surface in the first direction (①)) of the camera (340-1) faces away from the center of the lens assembly (320-1). For example, as shown in FIG. 7, when the camera (340-1) is positioned adjacent to the upper edge of the display panel (310), the front side (e.g., the surface in the first direction (①)) of the camera (340-1) may be arranged at an angle so that it faces upward. However, this is merely exemplary, and the angle and / or direction of inclination of the camera (340-1) are not limited thereto. Depending on the installation location of the camera (340-1) and / or the optical structure of the lens assembly (320-1), the angle and / or direction of inclination of the camera (340-1) may be set in various ways.

[0112] In one embodiment, the lens assembly (320-1) may be configured to provide a third infrared path (R3) through which infrared light irradiated from an infrared light source (e.g., the infrared light source (330) of FIG. 4A and / or the infrared light source (330) of FIG. 4B) reaches the user's eye (E) and a fourth infrared path (R4) through which infrared light reflected from the user's eye (E) reaches the camera (340-1). For example, when the camera (340-1) is positioned adjacent to one edge (e.g., the upper edge) of the display panel (310), the fourth infrared path (R4) may be formed in one area (e.g., the upper area) of the lens assembly (320-1). However, the third infrared path (R3) and the fourth infrared path (R4) illustrated in FIG. 7 are simplified for convenience of explanation, and the third infrared path (R3) and the fourth infrared path (R4) are not limited thereto.

[0113] In one embodiment, by positioning the camera (340-1) adjacent to the display panel (310), the camera (340-1) can be positioned in the direction of the user's gaze, and accordingly, the performance of the camera (340-1) in sensing the image of the user's eye (E) can be improved. For example, since the user wears the wearable electronic device (e.g., the wearable electronic device (200) of FIG. 2B) so that the eye (E) faces the display panel (310), the camera (340-1) positioned adjacent to the display panel (310) can be positioned in the direction of the user's gaze. Compared to the comparative example in which the camera is positioned outside the display assembly (e.g., outside the housing of the wearable electronic device), in the wearable electronic device (200) according to one embodiment, since the camera (340-1) is positioned in the direction of the user's gaze, the camera (340-1) can sense the image of the user's eye (E) more easily and accurately. In addition, since the camera (340-1) is positioned adjacent to the display panel (310), iris recognition can be performed while the user's gaze is directed toward the front. For example, in the comparative example where the camera is positioned outside the display assembly (e.g., outside the housing of the wearable electronic device), the user may have to turn his gaze outward and gaze at the outside of the display assembly (e.g., outside the housing of the wearable electronic device) for iris recognition. Compared to this comparative example, in the wearable electronic device (200) according to one embodiment, since the camera (340-1) is positioned in the direction of the user's gaze, iris recognition can be performed while the user is looking toward the front, thereby improving user convenience.

[0114]

[0115] In one embodiment, a wearable electronic device (200) may include a housing (210) configured to be worn on a user's head, a display panel (310) positioned in the housing (210) to generate visible light in at least a first direction, a lens assembly (320) positioned on the first direction side of the display panel (310), at least one infrared light source (330) positioned in the display panel (310) to irradiate infrared light toward an eye (E) of the user, and a camera (340) positioned on a second direction side of the display panel (310) opposite to the first direction to sense infrared light reflected from the eye (E) of the user. The camera (340) may be positioned outside an effective viewing area (F) of the display panel (310).

[0116] In one embodiment, the camera (340) may be positioned on the second direction side of the display panel (310) using an under display camera (UDC) structure.

[0117] In one embodiment, the infrared light source (330) may be positioned in the bezel area (B) of the display panel (310).

[0118] In one embodiment, the infrared light source (330) may be configured as part of a pixel of the display panel (310).

[0119] In one embodiment, the infrared light source (330) may be positioned adjacent to the outer edge of the display panel (310).

[0120] In one embodiment, the camera (340) may include a barrel (341), an image sensor (342) positioned inside the barrel (341), a first infrared filter (343) positioned on the first direction side of the barrel (341), and a second infrared filter (344) positioned on the first direction side of the image sensor (342).

[0121] In one embodiment, the lens assembly (320) may be configured to provide a first infrared path (R1) through which infrared rays irradiated from the infrared light source (330) reach the user's eye (E) and a second infrared path (R2) through which infrared rays reflected from the user's eye (E) reach the camera (340).

[0122] In one embodiment, the lens assembly (320) may be configured to provide a visible light path (V) through which visible light generated from the display panel (310) reaches the user's eye (E).

[0123] In one embodiment, the visible light path (V) and the second infrared path (R2) may not overlap each other.

[0124] In one embodiment, the wearable electronic device (200) may further include a polarizing plate for forming the first infrared path (R1) or the second infrared path (R2).

[0125] In one embodiment, the polarizing plate may be positioned on the first direction side of the lens assembly (320).

[0126] In one embodiment, the wearable electronic device (200) may further include a sensor position correction unit configured to correct the position of the image sensor (342) in response to shaking of the wearable electronic device (200).

[0127] In one embodiment, the wearable electronic device (200) may further include a processor configured to track the user's gaze or recognize the user's iris using sensing information of the camera (340).

[0128] In one embodiment, the camera (340) may be configured with at least 4 million pixels.

[0129] In one embodiment, the area (311) of the display panel (310) where the camera (340) is positioned may have an infrared transmittance of 40% or more.

[0130] In one embodiment, a display assembly (300) configured to be applied to a wearable electronic device (200) may include a display panel (310) that generates visible light in at least a first direction, a lens assembly (320) positioned on the first direction side of the display panel (310), at least one infrared light source (330) positioned on the display panel (310) that irradiates infrared light toward a user's eye (E), and a camera (340) positioned on a second direction side of the display panel (310) opposite to the first direction, that senses infrared light reflected from the user's eye (E). The camera (340) may be positioned outside an effective viewing area (F) of the display panel (310).

[0131] In one embodiment, the camera (340) may be positioned on the second direction side of the display panel (310) using an under display camera (UDC) structure.

[0132] In one embodiment, the infrared light source (330) may be positioned in the bezel area (B) of the display panel (310).

[0133] In one embodiment, the infrared light source (330) may be configured as part of a pixel of the display panel (310).

[0134] In one embodiment, a wearable electronic device (200) may include a housing (210) configured to be worn on a user's head, a display panel (310) positioned in the housing (210) to generate visible light toward at least a first direction, a lens assembly (320) positioned on the first direction side of the display panel (310), a polarizing plate positioned on the first direction side of the lens assembly (320), at least one infrared light source (330) positioned in the display panel (310) to irradiate infrared light toward an eye (E) of the user, and a camera (340) positioned on a second direction side of the display panel (310) opposite to the first direction to sense infrared light reflected from the eye (E) of the user. The camera (340) may be positioned on the second direction side of the display panel (310) using an under display camera (UDC) structure outside the effective viewing area (F) of the display panel (310). The lens assembly (320) may be configured to provide a first infrared path (R1) through which infrared rays irradiated from the infrared light source (330) reach the user's eye (E) and a second infrared path (R2) through which infrared rays reflected from the user's eye (E) reach the camera (340). The polarizing plate may form the first infrared path (R1) or the second infrared path (R2).

Claims

1. In a wearable electronic device (200), A housing (210) configured to be worn on a user's head; A display panel (310) positioned in the housing (210) so as to generate visible light toward at least a first direction; A lens assembly (320) positioned on the first direction side of the display panel (310); At least one infrared light source (330) positioned on the display panel (310) and irradiating infrared rays toward the user's eyes (E); and A camera (340) is positioned on the second direction side opposite to the first direction of the display panel (310) and senses infrared rays reflected from the user's eye (E). A wearable electronic device (200), wherein the above camera (340) is positioned outside the effective viewing area (F) of the display panel (310).

2. In paragraph 1, A wearable electronic device (200) in which the above camera (340) is positioned on the second direction side of the display panel (310) using a UDC (under display camera) structure.

3. In paragraph 1 or 2, The above infrared light source (330) is located in the bezel area (B) of the display panel (310) of the wearable electronic device (200).

4. In any one of paragraphs 1 to 3, A wearable electronic device (200), wherein the above infrared light source (330) is configured as a part of a pixel of the display panel (310).

5. In any one of paragraphs 1 to 4, The above infrared light source (330) is positioned adjacent to the outer edge of the display panel (310), a wearable electronic device (200).

6. In any one of paragraphs 1 to 5, The above camera (340) barrel (341); An image sensor (342) located inside the above barrel (341); A first infrared filter (343) positioned on the first direction side of the barrel (341); and A wearable electronic device (200) comprising a second infrared filter (344) positioned on the first direction side of the image sensor (342).

7. In any one of paragraphs 1 to 6, A wearable electronic device (200) in which the lens assembly (320) is configured to provide a first infrared path (R1) through which infrared rays irradiated from the infrared light source (330) reach the user's eye (E) and a second infrared path (R2) through which infrared rays reflected from the user's eye (E) reach the camera (340).

8. In any one of paragraphs 1 to 7, A wearable electronic device (200) in which the above lens assembly (320) is configured to provide a visible light path (V) through which visible light generated from the display panel (310) reaches the user's eye (E).

9. In any one of paragraphs 1 to 8, A wearable electronic device (200), wherein the visible light path (V) and the second infrared path (R2) do not overlap each other.

10. In any one of paragraphs 1 to 9, A wearable electronic device (200) further comprising a polarizing plate for forming the first infrared path (R1) or the second infrared path (R2).

11. In any one of paragraphs 1 to 10, A wearable electronic device (200), wherein the polarizing plate is positioned on the first direction side of the lens assembly (320).

12. In any one of paragraphs 1 to 11, A wearable electronic device (200) further comprising a sensor position correction unit configured to correct the position of the image sensor (342) in response to shaking of the wearable electronic device (200).

13. In any one of paragraphs 1 to 12, A wearable electronic device (200) further comprising a processor configured to track the user's gaze or recognize the user's iris using sensing information of the camera (340).

14. In any one of paragraphs 1 to 13, A wearable electronic device (200), wherein the above camera (340) is configured with at least 4 million pixels.

15. In any one of paragraphs 1 to 14, A wearable electronic device (200), wherein an area (311) of the display panel (310) where the camera (340) is positioned has an infrared transmittance of 40% or more.

Images