Head-mounted display device and display control method

Abstract

The head-mounted display device of the present disclosure comprises: a main body; a wearing structure coupled to the main body so that the main body may be seated on the head of a user; at least one camera; at least one sensor; a main display corresponding to the eyeballs of the user when the main body is worn by the user; a three-dimensional (3D) display facing the outside of the head-mounted display device when the main body is worn by the user; a memory storing instructions; and at least one processor, wherein the instructions, when individually or collectively executed by the at least one processor, cause the head-mounted display device to: detect, on the basis of the at least one sensor, whether the head-mounted display device is worn; determine, on the basis of the at least one sensor, whether a person is present; determine whether the person is present within a designated range; if the person is present within the designated range, determine whether the person is one person; if one person is present within the designated range, measure a first distance to the person on the basis of the at least one sensor; control a focal length of a lenticular lens or a lens array of the 3D display on the basis of the first distance; and display an image on the 3D display.

Description

Head-mounted display device and display control method

[0001] The present disclosure relates to a head mounted display device and a display control method.

[0002] Typically, a head-mounted display device (HMD) is mounted on a part of the body, such as the head, and functions to display images. Such a head-mounted device may be shaped like a pair of goggles or glasses.

[0003] The head mounted display device may further include an outward-facing 3D display.

[0004] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art related to the present disclosure.

[0005] The head mounted display device of the present disclosure may include a main body.

[0006] The head mounted display device of the present disclosure may include a wearing structure coupled to the main body to allow the main body to be placed on the user's head.

[0007] The head mounted display device of the present disclosure may include at least one camera.

[0008] The head mounted display device of the present disclosure may include at least one sensor.

[0009] The head mounted display device of the present disclosure may include a main display corresponding to the user's eyes when the main body is worn by the user.

[0010] The head mounted display device of the present disclosure may include a 3D (3-dimensional) display facing the outside of the head mounted display device when the main body is worn by the user.

[0011] The head mounted display device of the present disclosure may include a memory for storing instructions.

[0012] The head mounted display device of the present disclosure may include at least one processor.

[0013] The instructions of the present disclosure, when individually or collectively executed by the at least one processor, may cause the head mounted display device to detect wearing of the head mounted display device based on the at least one sensor.

[0014] The instructions of the present disclosure, when individually or collectively executed by the at least one processor, may cause the head mounted display device to detect at least one person in the vicinity of the user based on the at least one sensor.

[0015] The instructions of the present disclosure, when individually or collectively executed by the at least one processor, may cause the head mounted display device to determine whether at least one person is present within a specified range.

[0016] The instructions of the present disclosure, when individually or collectively executed by the at least one processor, may cause the head mounted display device to determine whether a person is present within the specified range.

[0017] The instructions of the present disclosure, when individually or collectively executed by the at least one processor, may cause the head mounted display device to measure a first distance to a person based on the at least one sensor, if a person is present within the specified range, control a focal length of a lenticular lens or lens array of the 3D display based on the first distance, and display an image on the 3D display.

[0018] The display control method of the head mounted display device of the present disclosure may include an operation of detecting wearing of the head mounted display device based on at least one sensor.

[0019] The display control method of the head mounted display device of the present disclosure may include an operation of detecting at least one person around the user based on at least one sensor.

[0020] A display control method of a head mounted display device of the present disclosure may include an operation of determining whether at least one person is present within a specified range.

[0021] The display control method of the head mounted display device of the present disclosure may include an operation of determining whether there is at least one person within the specified range.

[0022] A display control method of a head-mounted display device of the present disclosure may include an operation of measuring a first distance to a person based on at least one sensor when a person is present within the specified range, controlling a focal length of a lenticular lens or lens array of a 3D display based on the first distance, and displaying an image on the 3D display.

[0023] In connection with the description of the drawings, the same or similar reference numerals may be used for the same or similar components.

[0024] FIG. 1 is a block diagram of a head mounted display device in a network environment according to one embodiment of the present disclosure.

[0025] FIG. 2 is a drawing showing the front of a head mounted display device according to one embodiment of the present disclosure.

[0026] FIG. 3 is a drawing showing the back of a head mounted display device according to one embodiment of the present disclosure.

[0027] FIG. 4 is a diagram illustrating a display, a gaze tracking camera, and a waveguide according to one embodiment of the present disclosure.

[0028] FIG. 5 is a diagram illustrating a head mounted display device including a 3D (3-dimensional) display according to one embodiment of the present disclosure.

[0029] FIG. 6 is a diagram illustrating a 3D display according to one embodiment of the present disclosure.

[0030] FIG. 7 is a diagram illustrating a method for controlling a 3D display according to one embodiment of the present disclosure.

[0031] FIG. 8 is a diagram illustrating a method for controlling a 3D display according to one embodiment of the present disclosure.

[0032] FIG. 9 is a flowchart illustrating a 3D display control method of a head mounted display device according to one embodiment of the present disclosure.

[0033] FIG. 10 is a flowchart illustrating a method for controlling a focal length of a 3D display according to one embodiment of the present disclosure.

[0034] FIG. 11 is a diagram illustrating a method for controlling a 3D display of a head mounted display device according to one embodiment of the present disclosure.

[0035] FIG. 12 is a diagram illustrating a method for controlling a 3D display of a head mounted display device according to one embodiment of the present disclosure.

[0036] FIG. 13 is a diagram showing a 3D display when viewed by an outsider using a head mounted display device according to one embodiment of the present disclosure.

[0037] FIG. 14 is a diagram showing a screen of an area that a user wearing a head mounted display device according to one embodiment of the present disclosure is looking at and a screen of an area that a user is not looking at.

[0038] In general, when implementing 3D images through a 3D display included in a head-mounted display device, it is difficult to realistically convey a sense of three-dimensionality because, unlike electronic devices (e.g., TVs) with a fixed viewing distance, the distance between the head-mounted display device and an outsider is variable.

[0039] A head mounted display device and a display control method according to one embodiment of the present disclosure are intended to control the focal length of a 3D display based on the distance between the head mounted display device and an outsider and the interaction between the outsider and a user.

[0040] A head-mounted display device and a display control method according to one embodiment of the present disclosure can provide a clear 3D image by variably controlling the focal length of a 3D display.

[0041] FIG. 1 is a block diagram of a head mounted display device (100) in a network environment according to one embodiment of the present disclosure.

[0042] Referring to FIG. 1, in a network environment, a head mounted display device (100) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network).

[0043] In one embodiment, the head mounted display device (100) may communicate with the electronic device (104) via a server (108).

[0044] In one embodiment, the head mounted display device (100) may include a processor (120), a memory (130), an input circuit (150), an audio output circuit (155), a display (160), an audio circuit (170), a sensor (176), an interface (177), a connection terminal (178), a haptic circuit (179), a camera (180), a power management circuit (188), a battery (189), a communication circuit (190), a subscriber identification circuit (196), or an antenna (197).

[0045] In one embodiment, the processor (120) may include at least one processor. The processor (120) may include processing circuitry.

[0046] In one embodiment, the head mounted display device (100) may have at least one of these components (e.g., the connection terminal (178)) omitted, or one or more other components added.

[0047] In one embodiment, some of the components of the head mounted display device (100) (e.g., sensor (176), camera (180), or antenna (197)) may be integrated into one component (e.g., display (160)).

[0048] In one embodiment, the processor (120) may control at least one other component (e.g., a hardware or software component) of the head mounted display device (100) connected to the processor (120) by executing, for example, software (e.g., a program (140)) and may perform various data processing or operations.

[0049] In one embodiment, as at least part of data processing or calculation, the processor (120) may store instructions or data received from another component (e.g., a sensor (176) or a communication circuit (190)) in volatile memory (132), process the instructions or data stored in volatile memory (132), and store resulting data in non-volatile memory (134).

[0050] In one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the head mounted display device (100) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0051] In one embodiment, the auxiliary processor (123) may control at least a portion of functions or states associated with at least one component of the head mounted display device (100) (e.g., the display (160), the sensor (176), or the communication circuit (190)), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application running) state.

[0052] In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as part of another functionally related component (e.g., a camera (180) or communication circuitry (190)).

[0053] In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. Such learning may be performed, for example, within the head-mounted display device (100) itself where the artificial intelligence model is executed, or may be performed through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers. The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0054] In one embodiment, the memory (130) may store various data used by at least one component (e.g., the processor (120) or the sensor (176)) of the head mounted display device (100). The data may include, for example, software (e.g., the program (140)) and input data or output data for commands related thereto. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0055] In one embodiment, the program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0056] In one embodiment, the input circuit (150) may receive commands or data to be used by a component of the head mounted display device (100) (e.g., a processor (120)) from an external source (e.g., a user) of the head mounted display device (100). The input circuit (150) may include, for example, a microphone, a mouse, a keyboard, keys (e.g., buttons), or a digital pen (e.g., a stylus pen).

[0057] In one embodiment, the audio output circuit (155) can output audio signals to the outside of the head-mounted display device (100). The audio output circuit (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0058] In one embodiment, the display (160) can visually present information to an external party (e.g., a user) of the head mounted display device (100). The display (160) can include, for example, a display, a holographic device, or a projector and control circuitry for controlling the device.

[0059] In one embodiment, the display (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the magnitude of a force generated by a touch.

[0060] In one embodiment, the audio circuit (170) can convert sound into an electrical signal, or vice versa.

[0061] In one embodiment, the audio circuit (170) may acquire sound through the input circuit (150), or output sound through the audio output circuit (155), or an external electronic device (e.g., electronic device (102)) (e.g., a speaker, headphone, case, or phone) directly or wirelessly connected to the head-mounted display device (100).

[0062] In one embodiment, the sensor (176) may detect an operating state (e.g., power or temperature) of the head mounted display device (100) or an external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. In one embodiment, the sensor (176) may include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0063] In one embodiment, the sensor (176) may include at least one of an IR sensor, a red green blue (RGB) sensor, or an image sensor.

[0064] In one embodiment, the interface (177) may support one or more designated protocols that may be used to connect the head mounted display device (100) directly or wirelessly with an external electronic device (e.g., electronic device (102)).

[0065] In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0066] In one embodiment, the electronic device (102) may be the same or a different type of device as the head mounted display device (100).

[0067] In one embodiment, the electronic device (102) may include at least some of the components included in the head mounted display device (100). The electronic device (102) may include, for example, memory, a processor, a battery, or power management circuitry. The memory included in the electronic device (102) may store commands, data, or programs.

[0068] In one embodiment, all or part of the operations performed on the head mounted display device (100) may be performed on the electronic device (102). For example, when the head mounted display device (100) is to perform a function or service automatically or in response to a request from a user or another device, the head mounted display device (100) may, instead of or in addition to performing the function or service on its own, request one or more external electronic devices (e.g., the electronic device (102)) to perform the function or at least part of the service. The one or more external electronic devices (e.g., the electronic device (102)) that receive the request may perform at least a part of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the head mounted display device (100). The head mounted display device (100) may process the result, either as is or additionally, and provide it as at least part of a response to the request. For example, the electronic device (102) renders content data executed in an application and transmits it to the head mounted display device (100), and the head mounted display device (100) that receives the data can output the content data to the display (160). If the head mounted display device (100) detects user movement through an IMU sensor or the like, the processor (120) of the head mounted display device (100) can correct the rendering data received from the electronic device (102) based on the movement information and output it to the display (160). Alternatively, the head mounted display device (100) can transmit the movement information to the electronic device (102) and request rendering so that the screen data is updated accordingly.

[0069] In one embodiment, the electronic device (102) may be a device of various forms, such as a case device capable of storing and charging the head mounted display device (100).

[0070] In one embodiment, the connection terminal (178) may include a connector through which the head mounted display device (100) may be physically connected to an external electronic device (e.g., electronic device (102)).

[0071] In one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0072] In one embodiment, the haptic circuit (179) may convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that a user can perceive through a tactile or kinesthetic sense. In one embodiment, the haptic circuit (179) may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0073] In one embodiment, the camera (180) can capture still images and video. In one embodiment, the camera (180) can include one or more lenses, image sensors, image signal processors, or flashes.

[0074] In one embodiment, the head mounted display device (100) may include at least one camera (180). For example, the at least one camera (180) included in the head mounted display device (100) may include a camera for acquiring a real-world image facing the outside of the head mounted display device (100) and a camera for tracking the eyes of a wearer of the head mounted display device (100).

[0075] In one embodiment, the power management circuit (188) can manage power supplied to the head mounted display device (100). The power management circuit (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0076] In one embodiment, a battery (189) may power at least one component of the head mounted display device (100). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0077] In one embodiment, the communication circuit (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the head mounted display device (100) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication circuit (190) may operate independently of the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication.

[0078] In one embodiment, the communication circuit (190) may include wireless communication circuit (192) (e.g., cellular communication circuit, short-range wireless communication circuit, or global navigation satellite system (GNSS) communication circuit) or wired communication circuit (194) (e.g., local area network (LAN) communication circuit, or power line communication circuit). Any of these communication circuits may communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication circuits may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication circuit (192) can identify or authenticate the head mounted display device (100) within a communication network, such as the first network (198) or the second network (199), using subscriber information (e.g., international mobile subscriber identity (IMSI)) stored in the subscriber identification circuit (196).

[0079] In one embodiment, the wireless communication circuit (192) can support a 5G network and next-generation communication technologies after a 4G network, such as new radio access technology (NR). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimizing terminal power and connecting multiple terminals (mMTC (massive machine type communications)), or high-reliability and low-latency communications (URLLC (ultra-reliable and low-latency communications)). The wireless communication circuit (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication circuit (192) can support various technologies for securing performance in high-frequency bands, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication circuit (192) can support various requirements specified in the head-mounted display device (100), an external electronic device (e.g., electronic device (104)), or a network system (e.g., second network (199)).

[0080] In one embodiment, the wireless communication circuit (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB implementation, a loss coverage (e.g., 164 dB or less) for mMTC implementation, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL) each, or 1 ms or less for round trip) for URLLC implementation.

[0081] In one embodiment, the antenna (197) can transmit or receive signals or power to or from an external source (e.g., an external electronic device).

[0082] In one embodiment, the antenna (197) may include an antenna comprising a radiator formed of a conductor or conductive pattern formed on a substrate (e.g., a PCB).

[0083] In one embodiment, the antenna (197) may include a plurality of antennas (e.g., an array antenna). In such a case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication circuit (190). A signal or power may be transmitted or received between the communication circuit (190) and an external electronic device via the at least one selected antenna.

[0084] In one embodiment, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna (197) in addition to the radiator.

[0085] In one embodiment, the antenna (197) may form a mmWave antenna circuit.

[0086] In one embodiment, a mmWave antenna circuit may include a printed circuit board, an RFIC positioned on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) positioned on or adjacent a second side (e.g., a top side or a side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band.

[0087] In one embodiment, at least some of the components included in the head mounted display device (100) can be connected to each other and exchange signals (e.g., commands or data) with each other through a communication method between peripheral devices (e.g., a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)).

[0088] In one embodiment, commands or data may be transmitted or received between the head mounted display device (100) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or a different type of device as the head mounted display device (100).

[0089] In one embodiment, all or part of the operations executed by the head mounted display device (100) may be executed by one or more external electronic devices (102, 104, or 108). For example, when the head mounted display device (100) is to perform a certain function or service automatically or in response to a request from a user or another device, the head mounted display device (100) may, instead of executing the function or service itself or in addition, request one or more external electronic devices to perform at least a portion of the function or service. The one or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the head mounted display device (100). The head mounted display device (100) may process the result, either as is or additionally, and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technologies may be utilized, for example. The head-mounted display device (100) can provide ultra-low latency services, for example, using distributed computing or mobile edge computing. In another embodiment, the external electronic device (1804) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or neural networks.

[0090] In one embodiment, an external electronic device (104) or server (108) may be included in the second network (199). The head-mounted display device (100) may be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0091] FIG. 2 is a drawing showing the front of a head mounted display device (100) according to one embodiment of the present disclosure.

[0092] FIG. 3 is a drawing showing the back of a head mounted display device (100) according to one embodiment of the present disclosure.

[0093] Referring to FIGS. 2 and 3, a head mounted display device (100) can be worn on a part of a user's body to provide a user interface.

[0094] In one embodiment, the head-mounted display device (100) may output images and / or videos to the user. Alternatively, the head-mounted display device (100) may provide images related to augmented reality services and / or virtual reality services. For example, the head-mounted display device (100) may provide the user with experiences of augmented reality, virtual reality, mixed reality, and / or extended reality.

[0095] For example, a head-mounted display device (100) can provide augmented reality to a user. The head-mounted display device (100) can transmit a virtual object image output from the display (160) toward the user's eyes, and the virtual object image can utilize data regarding images of the real world captured through a plurality of cameras (230a, 230b, 230c).

[0096] In one embodiment, the head mounted display device (100) may be, for example, a head mounted display (HMD), a face mounted display (FMD), or may be, but is not limited to, smart glasses or a headset that provides extended reality such as augmented reality (AR), virtual reality glass (VR), or mixed reality.

[0097] In one embodiment, a head mounted display device (100) may include at least some of a housing (201), a plurality of cameras (230a, 230b, 230c), and a display (160).

[0098] In one embodiment, a head mounted display device (100) may include a housing (201). The housing (201) may be configured to accommodate at least one component. The housing (201) may include a first side (211a) (e.g., a front side), a second side (211b) opposite the first side (211a) (e.g., a back or wearing side), and a third side (211c) (e.g., a side side) between the first side (211a) and the second side (211b).

[0099] In one embodiment, the housing (201) may include a bridge (214). The bridge (214) may be configured to face a portion of the user's body, such as the nose. For example, the bridge (214) may be supported by the user's nose.

[0100] In one embodiment, the housing (201) may correspond to the main body of the head mounted display device (100). The housing (201) may be identical to the main body of the head mounted display device (100). The housing (201) may include the main body of the head mounted display device (100).

[0101] In one embodiment, the housing (201) may be mounted on the user's head by a wearable structure such as a template or strap.

[0102] In one embodiment, the head mounted display device (100) may include a lens structure (210, 220). The lens structure (210, 220) may include a plurality of lenses configured to adjust the focus of an image provided to a user. For example, the plurality of lenses may be configured to adjust the focus of an image output by the display (160). The plurality of lenses may be positioned at positions corresponding to positions of the display (160). The plurality of lenses may include, for example, a Fresnel lens, a pancake lens, a multi-channel lens, and / or any other suitable lens.

[0103] In one embodiment, the display (160) may be positioned corresponding to the lens structure (210, 220).

[0104] In one embodiment, the head mounted display device (100) may include a display (160). The display (160) may be configured to provide an image (e.g., a virtual image) to a user. For example, the display (160) may include a liquid crystal display (LCD), a digital mirror device (DMD), a liquid crystal on silicon (LCoS), an organic light emitting diode (OLED), and / or a micro light emitting diode (micro LED).

[0105] In one embodiment, when the display (160) includes at least one of a liquid crystal display (LCD), a digital mirror display, or a silicon liquid crystal display, the head mounted display device (100) may include a light source that irradiates light onto a screen output area of ​​the display (160).

[0106] In one embodiment, if the display (160) can generate light on its own, for example, if the display (160) includes at least one of an organic light emitting diode or a micro LED, the head mounted display device (100) can provide a good quality virtual image to the user even without including a separate light source.

[0107] In one embodiment, if the display (160) includes an organic light emitting diode or micro LED, a light source is unnecessary, and thus the head mounted display device (100) can be made lightweight. The head mounted display device (100) can include the display (160) and at least one transparent member. A user can use the head mounted display device (100) while wearing it on their face. The at least one transparent member can be formed of a glass plate, a plastic plate, or a polymer, and can be manufactured to be transparent or translucent.

[0108] In one embodiment, at least one transparent member may be positioned to face the user's right or left eye.

[0109] In one embodiment, if the display (150) is transparent, it may be positioned so as to face the user's eyes and form a screen display unit. In one embodiment, the display (160) may include a light source (not shown) configured to transmit a light signal to an area where an image is output.

[0110] In one embodiment, the display (160) can provide images to the user by generating optical signals on its own.

[0111] In one embodiment, the display (150) may be positioned on the second side (211b) of the housing (201). For example, one side of a pair of lenses of the display (160) may be positioned so that it is exposed to the outside through the second side (211b).

[0112] In one embodiment, the display (160) may be composed of organic light emitting diodes (OLEDs). For example, OLEDs can express red (R, red), green (G, green), and blue (B, blue) through self-luminescence of organic materials. However, the present invention is not limited thereto, and one pixel may include R, G, and B, and one chip may be implemented with multiple pixels including R, G, and B.

[0113] In one embodiment, the display (160) can display various images. Here, the image is a concept including still images and moving images, and the display (160) can display various images such as broadcast content, multimedia content, etc. In addition, the display (160) can also display a user interface (UI) and icons.

[0114] In one embodiment, the display (160) includes a separate IC chip, and the IC chip can display an image based on an image signal received from the processor (120). In one embodiment, the IC chip can generate driving signals for a plurality of light-emitting elements based on the image signal received from the processor (120), and display an image by controlling the light emission of a plurality of pixels included in the display panel based on the driving signals.

[0115] In one embodiment, the display (160) may include a plurality of pixels for displaying a virtual image. The display (160) may further include infrared pixels that emit infrared light.

[0116] In one embodiment, the display (160) may further include a photo-receiving pixel (e.g., a photo sensor pixel) disposed between the pixels and configured to receive light reflected from a user's eyes, convert it into electrical energy, and output it. The photo-receiving pixel may be referred to as a "gaze tracking sensor." The gaze tracking sensor may detect infrared light reflected by the user's eyes from light emitted by infrared pixels included in the display (160).

[0117] In one embodiment, the head mounted display device (100) can detect the user's gaze direction (e.g., eye movement) through light-receiving pixels.

[0118] In one embodiment, the head mounted display device (100) may determine the location of the center of the virtual image based on the gaze directions of the user's left and right eyes (e.g., the direction in which the pupils of the user's left and right eyes are looking) detected through one or more light-receiving pixels.

[0119] In one embodiment, the head mounted display device (100) may include at least one display. The head mounted display device (100) may include a display (160) as a main display and a 3D display (3D display (510) of FIG. 5) as a secondary display or an external display.

[0120] In one embodiment, the display (160) may include a condenser lens and / or a transparent waveguide. For example, the transparent waveguide may be at least partially positioned on a portion of the glass.

[0121] In one embodiment, light emitted from the display (160) may be incident on one end of the glass, and the incident light may be transmitted to the user through a waveguide and / or waveguide (e.g., waveguide) formed within the glass. The waveguide may be made of glass, plastic, or polymer, and may include a nano-pattern formed on one inner or outer surface, for example, a grating structure having a polygonal or curved shape.

[0122] In one embodiment, the incident light can be propagated or reflected within the waveguide by the nano-pattern and provided to the user.

[0123] In one embodiment, the waveguide may include at least one diffractive element (e.g., a diffractive optical element (DOE), a holographic optical element (HOE)) or at least one reflective element (e.g., a reflective mirror).

[0124] In one embodiment, the waveguide can guide display light emitted from a light source to a user's eye using at least one diffractive element or reflective element.

[0125] In one embodiment, the waveguide serves to transmit light generated by the display to the user's eyes.

[0126] In one embodiment, the waveguide may be made of glass, plastic, or polymer and may include nano-patterns formed on some of its internal or external surfaces, for example, a grating structure having a polygonal or curved shape.

[0127] In one embodiment, light incident on one end of the waveguide can be propagated inside the display (140) optical waveguide by the nano-pattern and provided to the user. In addition, the optical waveguide composed of a free-form prism can provide the incident light to the user through a reflective mirror. The optical waveguide can include at least one diffractive element (e.g., a Diffractive Optical Element (DOE), a Holographic Optical Element (HOE)) or at least one reflective element (e.g., a reflective mirror). The optical waveguide can guide display light emitted from a light source unit to the user's eyes by using at least one diffractive element or reflective element included in the optical waveguide.

[0128] In one embodiment, the diffractive element may include an input optical member / output optical member (not shown). For example, the input optical member may mean an input grating area, and the output optical member (not shown) may mean an output grating area. The input grating area may serve as an input terminal that diffracts (or reflects) light output from (e.g., a Micro LED) to transmit the light to a transparent member (e.g., a first transparent member, a second transparent member) of a screen display unit. The output grating area may serve as an outlet that diffracts (or reflects) light transmitted to a transparent member (e.g., a first transparent member, a second transparent member) of a waveguide to a user's eye.

[0129] In one embodiment, the reflective element may comprise a total internal reflection (TIR) ​​optical element or waveguide for total internal reflection. For example, total internal reflection may refer to a method of guiding light such that light (e.g., a virtual image) entering through the input grating region is 100% reflected from one surface (e.g., a specific surface) of the waveguide, thereby transmitting 100% of the light to the output grating region.

[0130] In one embodiment, light emitted from the display (160) may be guided along an optical path through an input optical element into a waveguide. Light traveling within the waveguide may be guided toward the user's eyes through an output optical element. The screen display may be determined based on the light emitted toward the user's eyes.

[0131] In one embodiment, the head mounted display device (100) may include a sensor (176). The sensor (176) may be configured to detect the depth of a subject. The sensor (176) may be configured to transmit a signal toward the subject and / or receive a signal from the subject. For example, the transmitted signal may include near infrared, ultrasound, and / or laser. The sensor (176) may be configured to measure the time of flight (ToF) of the signal to measure the distance between the head mounted display device (100) and the subject. The sensor (176) may be disposed on a first surface (211a) of the housing (201).

[0132] In one embodiment, the sensor (176) may include a depth sensor. The depth sensor may be used to determine the distance to an object. The depth sensor (e.g., the depth sensor (235) of FIG. 2) may include ToF (Time of Flight) technology. ToF technology may include technology that measures the distance to an object using signals (such as near-infrared, ultrasound, or laser). ToF technology may measure the time of flight of a signal by transmitting a signal and measuring the signal at a receiving unit.

[0133] In one embodiment, the camera (180) of FIG. 1 may include multiple cameras (230a, 230b, 230c).

[0134] In one embodiment, the plurality of cameras (230a, 230b, 230c) may include at least some of the first camera (230a), the second camera (230b), or the third camera (230c). The plurality of cameras (230a, 230b, 230c) may capture images of the exterior of the housing (201), for example, a user and / or other subjects. For example, the plurality of cameras (230a, 230b, 230c) may convert optical signals into input data and provide the data to the processor (120). In one embodiment, the processor (120) may receive the received input data and transmit output data to the display (160). The processor (120) may combine data received from each of the plurality of cameras (230a, 230b, 230c) and process the combined data to control the display (160).

[0135] In one embodiment, a first camera (230a) including at least one camera for capturing and a second camera (230b) including at least one camera for recognizing are spaced apart from a first surface (211a) of the housing (201) so as to capture a direction in which the first surface (211a) of the housing (201) faces.

[0136] In one embodiment, the camera (180) of FIG. 1 may include at least some of a first camera (230a), a second camera (230b), or a third camera (230c).

[0137] In one embodiment, the first camera (230a) and the second camera (230b) may be spaced apart from each other on the first side (211a) of the housing (201). The first camera (230a) and the second camera (230b) may be positioned to face different directions to capture various directions, such as the first side (211a) or the third side (211c).

[0138] In one embodiment, the first camera (230a) may be configured to acquire an image from a subject. The first camera (230a) may be formed in plurality, and one of the plurality of first cameras (230a) may be disposed in a portion of the first surface (211a) of the housing (201), and another first camera (230a) may be disposed in a portion of the first surface (211a) of the housing (201) and another portion of the housing (201).

[0139] In one embodiment, a plurality of first cameras (230a) may be positioned on each side of the depth sensor (235). The plurality of first cameras (230a) may include an image stabilizer actuator (not shown) and / or an autofocus actuator (not shown). For example, the plurality of first cameras (230a) may include at least one of a camera configured to acquire a color image, a global shutter camera, a rolling shutter camera, or a combination thereof.

[0140] In one embodiment, the second camera (230b) may be configured to recognize a subject. The second cameras (230b) may be formed in multiple numbers, and the multiple second cameras (230b) may be configured to detect and / or track an object (e.g., a human head or hand) or space with three or six degrees of freedom. For example, the multiple second cameras (230b) may include global shutter cameras. The multiple second cameras (1530b) may be configured to perform simultaneous localization and mapping (SLAM) using depth information of the subject. The multiple second cameras (230b) may be configured to recognize gestures of the subject.

[0141] In one embodiment, a plurality of second cameras (230b) may be arranged on the first side (211a) of the housing (201).

[0142] In one embodiment, the first camera (230a) and the second camera (230b) may be cameras for taking pictures, and may be referred to as high resolution (HR) or photo video (PV) cameras, and may include high-resolution cameras. The first camera (230a) and the second camera (230b) may include a color camera equipped with functions for obtaining high-quality images, such as an auto focus (AF) function and an optical image stabilizer (OIS). However, the present invention is not limited thereto, and the first camera (230a) and the second camera (230b) may include a global shutter (GS) camera or a rolling shutter (RS) camera.

[0143] In one embodiment, the head-mounted display device (100) may include a plurality of third cameras (230c). The plurality of third cameras (230c) may be configured to recognize a user's face. For example, the plurality of third cameras (230c) may be configured to detect and track a user's facial expressions.

[0144] In one embodiment, the third camera (230c) may include at least one camera for facial recognition or at least one camera for eye tracking.

[0145] In one embodiment, the head-mounted display device (100) may further include an eye tracking camera in at least some of the plurality of third cameras (230c). The eye tracking camera may be used to detect and track eye movements.

[0146] In one embodiment, the third camera (230c) can detect and track the pupil. The third camera (230c) can include multiple cameras corresponding to the left and right eyes.

[0147] In one embodiment, at least one of the plurality of cameras (230a, 230b, 230c) may include a camera used for 3 degrees of freedom (DoF), 6 degrees of freedom (DoF) head tracking, hand detection and tracking, gesture and / or spatial recognition.

[0148] In one embodiment, at least one of the plurality of cameras (230a, 230b, 230c) may include a global shutter (GS) camera for detecting and tracking head and hand movements. For example, a stereo camera may be used for head tracking and spatial recognition, so two global shutter (GS) cameras of the same specification and performance may be used, and a rolling shutter (RS) camera may be used to detect and track fine movements such as rapid hand and finger movements.

[0149] In one embodiment, at least one of the plurality of cameras (230a, 230b, 230c) may be primarily, but not necessarily limited to, a global shutter (GS) camera having superior camera performance (e.g., image drag) compared to other cameras, and for example, a rolling shutter (RS) camera may be used. At least one of the plurality of cameras (230a, 230b, 230c) may perform spatial recognition for six degrees of freedom (DoF) and simultaneous localization and mapping (SLAM) functions through depth capturing. At least one of the plurality of cameras (230a, 230b, 230c) may also perform a user gesture recognition function.

[0150] In one embodiment, the head-mounted display device (100) may include an inertial measurement unit (IMU) sensor. The IMU sensor may include at least one of an acceleration sensor, a gyroscope, and a magnetometer. The head-mounted display device (100) may detect a user's movements based on the IMU sensor.

[0151] In one embodiment, although not shown in the drawing, the head mounted display device (100) may include at least some of a sensor (not shown), a lighting unit (not shown), a plurality of microphones (not shown), a plurality of speakers (not shown), a battery (not shown), and a printed circuit board (not shown).

[0152] In one embodiment, the sensors (not shown) may be present in one or more configurations for various purposes (e.g., a gyro sensor, an acceleration sensor, a geomagnetic sensor, and / or a gesture sensor), and for example, the sensors (not shown) may perform at least one of head tracking for six degrees of freedom (DoF), pose estimation & prediction, gesture and / or spatial recognition, and / or slam functionality with depth imaging.

[0153] In one embodiment, the lighting unit (not shown) may have various uses depending on the location to which it is attached. For example, the lighting unit (not shown) may be attached around the second side (211b) of the head-mounted display device (100). The lighting unit (not shown) may be used as an auxiliary means to facilitate eye gaze detection when the gaze tracking camera (not shown) captures the pupil. The lighting unit (not shown) may use an infrared light emitting device (IR LED) of a visible light wavelength or an infrared wavelength.

[0154] For example, a lighting unit (not shown) may be attached to the front (211a) of the head-mounted display device (1200) or its surroundings. The lighting unit (not shown) may be used as a means to supplement the surrounding brightness when multiple front cameras (230a, 230b) are shooting. The lighting unit (not shown) may be used particularly in dark environments or when it is difficult to detect a subject to be shot due to mixed or reflected light from multiple light sources.

[0155] In one embodiment, the lighting unit (not shown) may be omitted. The lighting unit (not shown) may be replaced by infrared pixels included in the display (140). The lighting unit (not shown) may also be included in the head mounted display device (100) to assist the infrared pixels included in the display (160).

[0156] In one embodiment, multiple microphones (not shown) can process external acoustic signals into electrical voice data. The processed voice data can be utilized in various ways depending on the function (or application) being performed by the head-mounted display device (100).

[0157] In one embodiment, a plurality of speakers (not shown) can output audio data received from the communication circuit or stored in the memory (120).

[0158] In one embodiment, one or more batteries (not shown) may be included in the head mounted display device (100) and may supply power to components that make up the head mounted display device (100).

[0159] In one embodiment, a printed circuit board (not shown) can transmit electrical signals to each circuit (e.g., camera, display, audio, or sensor) and other printed circuit boards through a flexible printed circuit board (FPCB).

[0160] In one embodiment, a printed circuit board (not shown) may have control circuitry (not shown) positioned thereon that controls components that constitute a head mounted display device (100).

[0161] FIG. 4 is a diagram illustrating a display (160), a gaze tracking camera (410), and a waveguide (430) according to one embodiment of the present disclosure.

[0162] In one embodiment, the display (160) may include a waveguide (430). Light emitted from the display (160) may be transmitted to a user through the waveguide (430).

[0163] In one embodiment, the waveguide (430) may be made of glass, plastic, or polymer and may include nano-patterns formed on one surface of the inner or outer surface, for example, a grating structure having a polygonal or curved shape. The waveguide (430) may include a waveguide.

[0164] In one embodiment, light emitted from the display (160) is transmitted to the waveguide (430) through the input optical structure (451), and light input to the waveguide (430) can be propagated or reflected inside the waveguide (430) and provided to the user through the output optical structure (452).

[0165] In one embodiment, the waveguide (430) may include at least one diffractive element (e.g., a diffractive optical element (DOE), a holographic optical element (HOE)) or at least one reflective element (e.g., a reflective mirror).

[0166] In one embodiment, the waveguide (430) can guide light from the display (160) emitted from the light source to the user's eye using at least one diffractive element or reflective element.

[0167] In one embodiment, the waveguide (430) serves to transmit the light generated by the display (160) to the user's eyes.

[0168] In one embodiment, the waveguide (430) may be made of glass, plastic, or polymer and may include nano-patterns formed on some of its inner or outer surfaces, for example, a grating structure having a polygonal or curved shape.

[0169] In one embodiment, light incident on one end of the waveguide (430) can be propagated within the waveguide by the nano-pattern and provided to the user. Additionally, a waveguide composed of a free-form prism can provide the incident light to the user through a reflective mirror (e.g., an output optical structure (452)).

[0170] In one embodiment, the diffractive element may include an input optical structure (451) and an output optical structure (452). For example, the output optical structure (452) may include an input grating area. The output optical structure (452) may include an output grating area.

[0171] In one embodiment, the input grating region may act as an input terminal that diffracts (or reflects) light output from (e.g., a Micro LED) to transmit the light to a transparent member of the display unit.

[0172] In one embodiment, the output grating region may act as an outlet to diffract (or reflect) light transmitted to the transparent member of the waveguide (430) toward the user's eye.

[0173] In one embodiment, the reflective element may comprise a total internal reflection (TIR) ​​optical element or waveguide for total internal reflection. For example, total internal reflection may refer to a method of directing light such that light (e.g., a virtual image) entering through the input grating region is 100% reflected from one surface (e.g., a specific surface) of the waveguide (430) at an angle of incidence such that 100% of the light is transmitted to the output grating region.

[0174] In one embodiment, light emitted from the display (160) may be guided along an optical path through an input optical member to a waveguide (430). Light traveling within the waveguide (430) may be guided toward a user's eye through an output optical structure (452).

[0175] In one embodiment, the gaze tracking camera (410) may include the third camera (230c) of FIG. 3.

[0176] In one embodiment, a gaze tracking camera (410) may be used to detect and track pupils. The gaze tracking camera (410) may include a gaze tracking sensor (e.g., an infrared sensor) (411). The gaze tracking camera (410) may detect the user's pupils and track rapid eye movements through the gaze tracking sensor (e.g., an infrared sensor) (411). When light reflecting from the user's eyes is transmitted through an input structure (460) via a waveguide (440) for the gaze tracking camera, the light reflecting from the user's eyes may be transmitted to the gaze tracking camera (410) through the waveguide (440) for the gaze tracking camera.

[0177] FIG. 5 is a drawing showing a head mounted display device (100) including a 3D (3-dimensional) display (510) according to one embodiment of the present disclosure.

[0178] FIG. 6 is a diagram illustrating a 3D display (510) according to one embodiment of the present disclosure.

[0179] In one embodiment, the head mounted display device (100) of FIG. 5 may include components identical or similar to the head mounted display device (100) of FIG. 1, FIG. 2, or FIG. 3.

[0180] Referring to FIG. 5, the head mounted display device (100) may include a first camera (230a), a second camera (230b), a third camera (230c), a display (160), a depth sensor (235), and a 3D display (510).

[0181] In one embodiment, referring to FIGS. 2, 3 and 5, the first camera (230a), the second camera (230b), the depth sensor (235) or the 3D display (510) may be disposed on the first side (211a) of the head mounted display device (100), and the display (160) or the third camera (230c) may be disposed on the second side (211b) of the head mounted display device (100).

[0182] However, it is not limited thereto, and the third camera (230c) may be included in the lens structure (210, 220).

[0183] In one embodiment, at least one of the first camera (230a) or the second camera (230b) may include a pass through camera.

[0184] In one embodiment, the first camera (230a) may include at least one camera for capturing images. The second camera (230b) may include at least one camera for recognition.

[0185] In one embodiment, the first camera (230a) may be configured to acquire an image from a subject. The first camera (230a) may include a camera configured to acquire a color image.

[0186] In one embodiment, the second camera (230b) may be configured to recognize a subject. The second camera (230b) may be configured to detect and / or track an object (e.g., a human head or hand) or space with three or six degrees of freedom. The second camera (1530b) may be configured to perform simultaneous localization and mapping (SLAM) using depth information of the subject.

[0187] In one embodiment, the first camera (230a) and the second camera (230b) may be referred to as HR (high resolution) or PV (photo video) cameras and may include high-resolution cameras.

[0188] In one embodiment, the first camera (230a) and the second camera (230b) may include a color camera equipped with functions for obtaining high-quality images, such as an auto focus (AF) function and an optical image stabilizer (OIS). However, the present invention is not limited thereto, and the first camera (230a) and the second camera (230b) may include a global shutter (GS) camera or a rolling shutter (RS) camera.

[0189] In one embodiment, the third camera (230c) may include at least one camera for eye tracking. The third camera (230c) may be used to detect and track pupils. The third camera (230c) may detect and track pupils. The third camera (230c) may include multiple cameras corresponding to the user's left eye (511) and right eye (512).

[0190] In one embodiment, the display (160) may be positioned at a location corresponding to the lens structures (210, 220) of FIG. 3. The lens structures (210, 220) of FIG. 3 may include a plurality of lenses configured to adjust the focus of an image provided to a user. For example, the plurality of lenses may be configured to adjust the focus of an image output by the display (160). The plurality of lenses may be positioned at a location corresponding to the location of the display (160). The plurality of lenses may include, for example, a Fresnel lens, a pancake lens, a multi-channel lens, and / or any other suitable lens.

[0191] In one embodiment, the display (160) may be configured to provide an image (e.g., a virtual image) to the user.

[0192] In one embodiment, the depth sensor (235) can be used to determine the distance to an object. The depth sensor (235) can measure the distance to an object using a signal (near infrared, ultrasound, laser, etc.).

[0193] In one embodiment, the 3D display (510) may be positioned so that its screen faces the first side (211a) of the head mounted display device (100). While the display (160) faces the eyes of the wearer or user of the head mounted display device (100), the 3D display (510) may display its screen toward the outside of the head mounted display device (100) rather than toward the wearer.

[0194] Referring to FIGS. 5 and 6, the 3D display (510) can display a three-dimensional stereoscopic image. The 3D display (510) displays optically separated multi-view images, and uses a specific lens (e.g., a lenticular lens or lens array (602)) to project light corresponding to images from different viewpoints onto the user's left eye (511) and right eye (512), thereby allowing the user to experience a three-dimensional effect.

[0195] In one embodiment, the 3D display (510) may include a lenticular lens or lens array (602) in front of the panel (601) to allow the left eye (511) and the right eye (512) to see different pixels (L, R). For example, the lenticular lens may be an array of semi-cylindrical lenses, and the lens array may be an array of hemispherical lenses.

[0196] For example, in order for an outsider (501, 502) looking at a 3D display (510) to feel a sense of three-dimensionality, the outsider must be within a certain viewing distance. However, the viewing distance for providing a 3D image is not fixed because the head-mounted display device (100) moves according to the wearer's intention or due to the movement of the outsider (501, 502).

[0197] In one embodiment, the instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to identify an outsider (501) being gazed at by the user of the head mounted display device (100) using a third camera (230c) that tracks the gaze of the wearer.

[0198] In one embodiment, the instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to determine a distance (e.g., a first distance (D1)) between an outsider (501) identified based on the depth sensor (235) and the head mounted display device (100).

[0199] In one embodiment, the instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to control the focal length of a lens included in the 3D display (510) based on a distance (e.g., a first distance (D1)) between an outsider (501) and the head mounted display device (100), and to display an image or a 3D image on the 3D display (510).

[0200] In one embodiment, the focal length of the lens included in the 3D display (510) is changed based on the distance (e.g., the first distance (D1)) between an outsider (501) and the head mounted display device (100), so that the outsider (501) can feel a three-dimensional effect when looking at the head mounted display device (100).

[0201] FIG. 7 is a drawing showing a control method of a 3D display (510) according to one embodiment of the present disclosure.

[0202] FIG. 8 is a drawing showing a control method of a 3D display (510) according to one embodiment of the present disclosure.

[0203] Referring to FIGS. 6, 7 and 8, the 3D display (510) may include a lenticular lens or lens array (602) in front of the panel (601) to allow different pixels (L, R) to be viewed by the left eye (511) and the right eye (512).

[0204] In one embodiment, a lenticular lens or lens array (602) may include at least one lens (6021, 6022, 6023) having a variable focal length and at least one electrode (E1, E2, E3, E4, E5, E6) corresponding to the at least one lens (6021, 6022, 6023).

[0205] In one embodiment, instructions stored in memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to apply voltage to at least one electrode (E1, E2, E3, E4, E5, E6).

[0206] In one embodiment, at least one of the electrodes (E1, E2, E3, E4, E5, E6) may comprise a transparent electrode. At least one of the electrodes (E1, E2, E3, E4, E5, E6) may be positioned on each of the left and right sides and / or the top and bottom sides of the lenticular lens or lens array (602).

[0207] In one embodiment, when voltage is applied to at least one electrode (E1, E2, E3, E4, E5, E6), the curvature (R1, R2) of at least one lens (6021, 6022, 6023) can be changed.

[0208] In one embodiment, at least one lens (6021, 6022, 6023) may include a material whose curvature can be changed by an electrical stimulus (e.g., voltage). At least one lens (6021, 6022, 6023) may include a liquid lens whose curvature can be changed by an electrical stimulus (e.g., voltage).

[0209] In one embodiment, the focal length of the lenticular lens or lens array (602) or the focal length of at least one lens (6021, 6022, 6023) may be inversely proportional to the curvature (R1, R2).

[0210] Referring to FIGS. 7 and 8, the first curvature (R1) of the lenticular lens or lens array (602) or the first curvature (R1) of at least one lens (6021, 6022, 6023) may be greater than the second curvature (R2) of the lenticular lens or lens array (602) or the second curvature (R2) of at least one lens (6021, 6022, 6023).

[0211] In one embodiment, a lenticular lens or lens array (602) or at least one lens (6021, 6022, 6023) having a first curvature (R1) may have a longer focal length than a lenticular lens or lens array (602) or at least one lens (6021, 6022, 6023) having a second curvature (R2).

[0212] In one embodiment, the viewing distance of the 3D display (510) may be proportional to the focal length of the lenticular lens or lens array (602) or at least one lens (6021, 6022, 6023). For example, a greater focal length of the lenticular lens or lens array (602) or at least one lens (6021, 6022, 6023) may increase the viewing distance of the 3D display (510). The viewing distance may correspond to the distance between an outsider (501) and the head mounted display device (100).

[0213] FIG. 9 is a flowchart illustrating a method for controlling a 3D display (510) of a head mounted display device (100) according to one embodiment of the present disclosure.

[0214] In one embodiment, at operation 901, instructions stored in memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to detect a user's wearing of the head mounted display device (100).

[0215] For example, the head mounted display device (100) may include an inertial measurement unit (IMU) sensor. The IMU sensor may include at least one of an acceleration sensor, a gyroscope, and a magnetometer. The head mounted display device (100) may detect whether a user is wearing the head mounted display device (100) based on the IMU sensor. The head mounted display device (100) may detect whether a user is wearing the head mounted display device (100) based on a gaze tracking camera (e.g., the third camera (230c)). However, the present invention is not limited thereto, and the head mounted display device (100) may detect whether a user is wearing the head mounted display device (100) based on the IMU sensor and the gaze tracking camera (e.g., the third camera (230c)).

[0216] In one embodiment, at operation 903, the instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to detect a person in the vicinity of the user or the surroundings of the head mounted display device (100) based on at least one sensor (e.g., depth sensor (235)). However, the present invention is not limited thereto, and at operation 903, the instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to detect a person in the vicinity of the user or the surroundings of the head mounted display device (100) based on the first camera (230a).

[0217] In one embodiment, at operation 905, instructions stored in memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to determine whether a person is present within a specified range.

[0218] In one embodiment, if a person is present within a specified range, instructions stored in memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to branch from operation 905 to operation 907.

[0219] In one embodiment, if there is no person within the specified range, the instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to branch from operation 905 to operation 903.

[0220] In one embodiment, at operation 907, the instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to determine whether a person is present within a specified range.

[0221] In one embodiment, if it is determined that there is a person within a specified range, the instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to branch from operation 907 to operation 909.

[0222] In one embodiment, if it is determined that there are multiple people (e.g., two or more) within a specified range, instructions stored in memory (130) may cause the head mounted display device (100) to branch from operation 907 to operation 913 when individually or collectively executed by at least one processor (120).

[0223] In one embodiment, at operation 909, the instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to measure a first distance to a person based on at least one sensor (e.g., depth sensor (235)) when a person is present within a specified range, and to control a focal length of a lenticular lens or lens array (602) of the 3D display (510) based on the measured first distance.

[0224] In one embodiment, in operation 911, the instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to display an image on a 3D display (510) having a focal length of a lenticular lens or lens array (602) adjusted based on the measured first distance.

[0225] In one embodiment, in operation 913, instructions stored in memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to identify the wearer's actions or the actions of people when it is determined that there are multiple people (e.g., two or more people) within a specified range.

[0226] In one embodiment, at operation 915, the instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to identify one person among multiple people (e.g., two or more people) based on at least one of the wearer's actions or the people's actions.

[0227] In one embodiment, at operation 913, instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to estimate the gaze of a user of the head mounted display device (100) based on a third camera (230c) capable of tracking the gaze of the user.

[0228] In one embodiment, at operation 915, the instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to identify a person the user is gazing at among several people based on the tracked gaze of the user.

[0229] In one embodiment, at operation 913, instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to identify a conversation between a user of the head mounted display device (100) and an outsider (e.g., people within a specified range) based on an input circuit (150) (e.g., a microphone).

[0230] In one embodiment, at operation 915, the instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to identify one of several people based on conversations with the identified user and outsiders (e.g., people within a specified range). For example, the instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to identify an outsider with whom the user primarily converses.

[0231] In one embodiment, in operation 913, instructions stored in memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to identify at least one of the user's gaze or the user's conversation.

[0232] In one embodiment, at operation 915, the instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to identify one of several people based on at least one of the identified user's gaze or the user's conversation. For example, the instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to identify an outsider among several people whose gaze the user primarily gazes at while conversing.

[0233] In one embodiment, at operation 915, the instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to identify one of several people based on at least one of the identified user's gaze or the user's conversation.

[0234] In one embodiment, at operation 915, instructions stored in memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to specify a peripheral person being viewed by the user.

[0235] In one embodiment, at operation 915, the instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to specify a peripheral person the user is looking at, if multiple peripheral persons are looking at the user.

[0236] In one embodiment, at operation 915, the instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to specify a peripheral person with whom the user is conversing, if multiple peripheral persons are viewing the user.

[0237] In one embodiment, at operation 915, instructions stored in memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to identify a nearby person close to the user when multiple nearby persons are looking at the user.

[0238] For example, a user may include a person wearing a head mounted display device (100), and a person in the vicinity may include a person in the vicinity of a wearer (user) of the head mounted display device (100).

[0239] In one embodiment, at operation 915, the instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to identify one of multiple people or multiple surrounding people based on gaze, conversation, and / or a combination thereof.

[0240] In one embodiment, at operation 915, the instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to maintain a focal distance for a predetermined period of time even if gaze, conversation, and / or a combination thereof is released. For example, if a user briefly looks away and then looks back at a particular person in the vicinity, the instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to maintain a focal distance for a predetermined period of time without immediately changing the focal distance when the user looks away.

[0241] In one embodiment, in operation 913, instructions stored in memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to identify human actions.

[0242] In one embodiment, at operation 915, instructions stored in memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to specify one of several people based on a priority corresponding to the people's actions.

[0243] In one embodiment, at operation 915, the instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to identify a person among multiple people looking at the head mounted display device (100) based on at least one sensor (e.g., a depth sensor (235) or a second camera (230b).

[0244] In one embodiment, at operation 915, instructions stored in memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to specify a person among outsiders (e.g., people within a specified range) with whom the user is conversing.

[0245] In one embodiment, at operation 915, the instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to identify a person close to the head mounted display device (100) or the user of the head mounted display device (100) among multiple people based on at least one sensor (e.g., a depth sensor (235) or a second camera (230b).

[0246] In one embodiment, at operation 915, the instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to identify one of several people based on at least one of a conversation between the outsider, a distance between the outsider and the user, or whether the outsider is gazing at the head mounted display device (100).

[0247] In one embodiment, the head mounted display device (100) may store in memory (130) a priority regarding a process of specifying one person among several people. For example, the head mounted display device (100) may assign a priority based on user input or a preset priority for a conversation between an outsider, a distance between the outsider and the user, or whether the outsider is gazing at the head mounted display device (100), and store the assigned priority (or priority) in memory (130).

[0248] In one embodiment, at operation 917, the instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to measure a second distance to a specified person based on at least one sensor (e.g., a depth sensor (235)) and to control a focal length of a lenticular lens or lens array (602) of the 3D display (510) based on the second distance.

[0249] In one embodiment, at operation 919, the instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to display an image on a 3D display (510) having a focal length of a lenticular lens or lens array (602) adjusted based on the measured second distance.

[0250] In one embodiment, the first distance and the second distance may be different or the same.

[0251] FIG. 10 is a flowchart illustrating a method for controlling a focal distance of a 3D display (510) according to one embodiment of the present disclosure.

[0252] In one embodiment, in operation 1001, instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to measure a distance (e.g., the first distance or the second distance in FIG. 9) between the head mounted display device (100) and a person or a specific person based on at least one sensor (e.g., the depth sensor (235) or the second camera (230b)).

[0253] In one embodiment, in operation 1003, instructions stored in memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to calculate a focal distance to control a viewing distance corresponding to a distance (e.g., the first distance or the second distance of FIG. 9).

[0254] In one embodiment, at operation 1005, the instructions stored in the memory (130), when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to apply a voltage corresponding to the calculated focal length of the lenticular lens or lens array (602) to an electrode of the lenticular lens or lens array (602) (e.g., at least one electrode (E1, E2, E3, E4, E5, E6)).

[0255] In one embodiment, the memory (130) may store information about the focal length of a lenticular lens or lens array (602) corresponding to a distance (e.g., the first distance or the second distance of FIG. 9) or a viewing distance.

[0256] In one embodiment, the memory (130) may store information about the curvature of a lenticular lens or lens array (602) corresponding to a focal length.

[0257] In one embodiment, a lenticular lens or lens array (602) may include at least one lens (6021, 6022, 6023) having a variable focal length and at least one electrode (E1, E2, E3, E4, E5, E6) corresponding to the at least one lens (6021, 6022, 6023).

[0258] In one embodiment, at least one lens (6021, 6022, 6023) can have its curvature changed depending on a voltage applied to at least one electrode (E1, E2, E3, E4, E5, E6).

[0259] In one embodiment, the memory (130) may store information corresponding to the curvature or focal length of at least one lens (6021, 6022, 6023) corresponding to a voltage applied to at least one electrode (E1, E2, E3, E4, E5, E6).

[0260] FIG. 11 is a drawing illustrating a method for controlling a 3D display (510) of a head mounted display device (100) according to one embodiment of the present disclosure.

[0261] In one embodiment, the head mounted display device (100) may, under the control of at least one processor (120), use at least one sensor (e.g., a depth sensor (235) or a second camera (230b)) to determine whether a person (1103, 1105) is present within a specified range (W1).

[0262] In one embodiment, if a person is present within a specified range (W1), the head mounted display device (100) may, under the control of at least one processor (120), use at least one sensor (e.g., a depth sensor (235) or a second camera (230b)) to determine whether there is one person (1103) present within the specified range.

[0263] In one embodiment, if it is determined that a person (1103) is within a specified range, the head mounted display device (100) may, under the control of at least one processor (120), use at least one sensor (e.g., a depth sensor (235) or a second camera (230b)) to determine the distance (D2) to the specified person (1103).

[0264] In one embodiment, the head mounted display device (100) may, under the control of at least one processor (120), calculate a focal length of a lenticular lens or lens array (602) to control a viewing distance corresponding to the distance (D2).

[0265] In one embodiment, the head mounted display device (100) may, under the control of at least one processor (120), apply a voltage corresponding to the calculated focal length of the lenticular lens or lens array (602) to electrodes of the lenticular lens or lens array (602) (e.g., at least one electrode (E1, E2, E3, E4, E5, E6)).

[0266] In one embodiment, the head mounted display device (100) may, under the control of at least one processor (120), display an image or 3D image based on a 3D display (510) whose focal length is changed in response to a distance (D2) to a specified person (1103).

[0267] FIG. 12 is a drawing illustrating a method for controlling a 3D display (510) of a head mounted display device (100) according to one embodiment of the present disclosure.

[0268] In one embodiment, the head mounted display device (100) may, under the control of at least one processor (120), use at least one sensor (e.g., a depth sensor (235) or a second camera (230b)) to determine whether a person (1203, 1205) is present within a specified range (W1).

[0269] In one embodiment, when a person is present within a specified range (W1), the head mounted display device (100) can determine, under the control of at least one processor (120), whether there is one person or multiple people present within the specified range using at least one sensor (e.g., a depth sensor (235) or a second camera (230b)).

[0270] In one embodiment, if it is determined that there are multiple people within a specified range, the head mounted display device (100) can track the eyes (511, 512) of the user (1201) based on a third camera (230c) capable of tracking gaze under the control of at least one processor (120), and identify the person (1203) that the user (1201) is looking at.

[0271] In one embodiment, the head mounted display device (100) may, under the control of at least one processor (120), use at least one sensor (e.g., a depth sensor (235) or a second camera (230b)) to determine a distance (D4) to a specific person (1203).

[0272] In one embodiment, the head mounted display device (100) may, under the control of at least one processor (120), calculate a focal length of a lenticular lens or lens array (602) to control a viewing distance corresponding to the distance (D4).

[0273] In one embodiment, the head mounted display device (100) may, under the control of at least one processor (120), apply a voltage corresponding to the calculated focal length of the lenticular lens or lens array (602) to electrodes of the lenticular lens or lens array (602) (e.g., at least one electrode (E1, E2, E3, E4, E5, E6)).

[0274] In one embodiment, the head mounted display device (100) may, under the control of at least one processor (120), display an image or 3D image based on a 3D display (510) whose focal length is changed in response to a distance (D4) to a specified person (1103).

[0275] FIG. 13 is a drawing showing a 3D display (510) when viewed by an outsider using a head mounted display device (100) according to one embodiment of the present disclosure.

[0276] Screen 1301 shows a case where the focal length of the 3D display (510) of the head mounted display device (100) is changed to correspond to an outsider, and screen 1303 shows a case where the focal length of the 3D display (510) of the head mounted display device (100) does not correspond to an outsider.

[0277] On the 1301 screen, an outsider can clearly see the image or 3D image displayed on the 3D display (510).

[0278] On the 1303 screen, an outsider may see an image or 3D image displayed on the 3D display (510) as opaque.

[0279] FIG. 14 is a diagram showing a screen of an area that a user wearing a head mounted display device (100) according to one embodiment of the present disclosure is looking at and a screen of an area that a user is not looking at.

[0280] On screen 1401, the area that a user wearing a head mounted display device (100) is looking at can be displayed on the display (160) as a high-resolution image (1410).

[0281] On screen 1403, an area that is not being viewed by a user wearing a head mounted display device (100) may be displayed on the display (160) as a low-resolution image (1420).

[0282] In one embodiment, a head mounted display device (100) includes a main body (e.g., a housing (201)), a wearing structure coupled to the main body (e.g., the housing (201)) to allow the main body (e.g., the housing (201)) to be placed on the user's head, at least one camera (e.g., a first camera (230a), a second camera (230b), a third camera (230c)), at least one sensor (e.g., a depth sensor (235), an IMU sensor, or an eye tracking camera), a main display (e.g., a display (160)) corresponding to the user's eyes when the main body (e.g., the housing (201)) is worn by the user, a 3D (dimensional) display (510) facing the outside of the head mounted display device (100) when the main body (e.g., the housing (201)) is worn by the user, a memory (130) storing instructions, and at least one processor (120), wherein the instructions are executed by at least one processor (120). When executed individually or collectively, the head mounted display device (100) may be caused to detect wearing of the head mounted display device (100) based on at least one sensor (e.g., a depth sensor (235), an IMU sensor, or an eye tracking camera), detect a person around the user based on at least one sensor (e.g., a depth sensor (235), an IMU sensor, or an eye tracking camera), determine whether the person exists within a specified range, and if the person exists within the specified range, determine whether the person is one, and if the person exists within the specified range, measure a first distance to the person based on at least one sensor (e.g., a depth sensor (235), an IMU sensor, or an eye tracking camera), control a focal length of a lenticular lens or lens array (602) of a 3D display (510) based on the first distance, and display an image on the 3D display (510).

[0283] In one embodiment, the instructions, when executed individually or collectively by at least one processor (120), may cause the head mounted display device (100) to identify a user's action or actions of people if two or more people are present within a specified range, and to identify a person based on at least one of the user's action or actions of people.

[0284] In one embodiment, the instructions, when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to measure a second distance to a specified person based on at least one sensor (e.g., a depth sensor (235), an IMU sensor, or an eye tracking camera), control a focal length of a lenticular lens or lens array (602) of the 3D display (510) based on the second distance, and display an image on the 3D display (510).

[0285] In one embodiment, the instructions, when executed individually or collectively by at least one processor (120), may cause the head mounted display device (100) to track the gaze of a user based on at least one sensor (e.g., a depth sensor (235), an IMU sensor, or an eye tracking camera) and to identify a person the user is looking at based on the tracked gaze.

[0286] In one embodiment, the instructions, when executed individually or collectively by at least one processor (120), may cause the head mounted display device (100) to identify a user's conversation based on at least one sensor (e.g., a depth sensor (235), an IMU sensor, or an eye tracking camera) and to identify a person based on the conversation with the user.

[0287] In one embodiment, the instructions, when executed individually or collectively by at least one processor (120), may cause the head mounted display device (100) to identify a person's behavior based on at least one sensor (e.g., a depth sensor (235), an IMU sensor, or an eye tracking camera) and to specify the person based on a priority corresponding to the person's behavior.

[0288] In one embodiment, the instructions, when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to identify a person among people looking at the head mounted display device (100) based on at least one sensor (e.g., a depth sensor (235), an IMU sensor, or an eye tracking camera).

[0289] In one embodiment, the instructions, when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to identify, based on at least one sensor (e.g., a depth sensor (235), an IMU sensor, or an eye tracking camera), a person among people who is conversing with the user of the head mounted display device (100), or to identify, based on at least one sensor (e.g., a depth sensor (235), an IMU sensor, or an eye tracking camera), a person among people who is close to the user of the head mounted display device (100).

[0290] In one embodiment, the lenticular lens or lens array (602) may include at least one lens having a variable focal length, and at least one electrode corresponding to the at least one lens.

[0291] In one embodiment, the instructions, when individually or collectively executed by at least one processor (120), may cause the head mounted display device (100) to calculate a focal distance for controlling a viewing distance corresponding to a first distance, and to apply a voltage corresponding to the calculated focal distance to at least one electrode.

[0292] In one embodiment, a display control method of a head mounted display device (100) may include an operation of detecting wearing of the head mounted display device (100) based on at least one sensor (e.g., a depth sensor (235), an IMU sensor, or an eye tracking camera), an operation of detecting a person around the user based on at least one sensor (e.g., a depth sensor (235), an IMU sensor, or an eye tracking camera), an operation of determining whether a person exists within a specified range, an operation of determining whether there is one person if there is one person within the specified range, and an operation of measuring a first distance to the person based on at least one sensor (e.g., a depth sensor (235), an IMU sensor, or an eye tracking camera), and controlling a focal length of a lenticular lens or lens array (602) of a 3D display (510) based on the first distance, and an operation of displaying an image on the 3D display (510).

[0293] In one embodiment, a display control method of a head mounted display device (100) may include an action of confirming a user's action or a person's action when two or more people are present within a specified range, and an action of specifying a person based on at least one of the user's action or the person's action.

[0294] In one embodiment, a display control method of a head mounted display device (100) may include measuring a second distance to a specified person based on at least one sensor (e.g., a depth sensor (235), an IMU sensor, or an eye tracking camera), controlling a focal length of a lenticular lens or lens array (602) of a 3D display (510) based on the second distance, and displaying an image on the 3D display (510).

[0295] In one embodiment, a display control method of a head mounted display device (100) may include an operation of tracking a user's gaze based on at least one sensor (e.g., a depth sensor (235), an IMU sensor, or an eye tracking camera), and an operation of specifying a person the user is looking at based on the tracked gaze.

[0296] In one embodiment, a display control method of a head mounted display device (100) may include an operation of confirming a user's conversation based on at least one sensor (e.g., a depth sensor (235), an IMU sensor, or an eye tracking camera), and an operation of identifying a person based on the conversation with the user.

[0297] In one embodiment, a display control method of a head mounted display device (100) may include an operation of identifying a person's behavior based on at least one sensor (e.g., a depth sensor (235), an IMU sensor, or an eye tracking camera), and an operation of specifying a person based on a priority corresponding to the person's behavior.

[0298] In one embodiment, a display control method of a head mounted display device (100) may include an operation of identifying a person among people looking at the head mounted display device (100) based on at least one sensor (e.g., a depth sensor (235), an IMU sensor, or an eye tracking camera).

[0299] In one embodiment, a display control method of a head mounted display device (100) may include an operation of specifying a person among people who is talking to a user of the head mounted display device (100) based on at least one sensor (e.g., a depth sensor (235), an IMU sensor, or an eye tracking camera), and an operation of specifying a person among people who is close to the user of the head mounted display device (100) based on at least one sensor (e.g., a depth sensor (235), an IMU sensor, or an eye tracking camera).

[0300] In one embodiment, a display control method of a head mounted display device (100) may include an operation of calculating a focal distance for controlling a viewing distance corresponding to a first distance, and an operation of applying a voltage corresponding to the calculated focal distance to at least one electrode.

[0301] An electronic device according to an embodiment disclosed in this document may take various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance. The electronic device according to an embodiment of this document is not limited to the aforementioned devices.

[0302] It should be understood that the embodiments of this document and the terminology used herein are not intended to limit the technical features described in this document to specific embodiments, but include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0303] The term "module" used in one embodiment of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0304] An embodiment of the present document may be implemented as software (e.g., a program) including one or more instructions stored in a storage medium (e.g., built-in memory or external memory) readable by a machine (e.g., an electronic device (100)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (100)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0305] According to one embodiment, the method according to one embodiment disclosed in the present document may be provided as a computer program product. The computer program product may be traded between sellers and buyers as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or may be provided through an application store (e.g., Play Store). TM ) or directly between two user devices (e.g., smart phones), online distribution (e.g., downloading or uploading). In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0306] According to one embodiment, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to one embodiment, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to one embodiment, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In a head-mounted display device, entity; A wearing structure that is coupled to the main body and allows the main body to be placed on the user's head; At least one camera; At least one sensor; When the above body is worn by a user, a main display corresponding to the user's eyes; A 3D (dimensional) display facing the outside of the head mounted display device when the main body is worn by the user; Memory that stores instructions; Contains at least one processor, The above instructions, when individually or collectively executed by the at least one processor, cause the head mounted display device to: Based on at least one sensor, detecting the wearing of the head mounted display device, Based on at least one sensor, detect a person around the user, Determine whether a person exists within a specified range, If there is a person within the above specified range, determine whether there is one person or not. A head mounted display device that measures a first distance to a person based on at least one sensor when a person exists within the above-mentioned specified range, controls a focal length of a lenticular lens or lens array of the 3D display based on the first distance, and displays an image on the 3D display.

2. In paragraph 1, The above instructions, when individually or collectively executed by the at least one processor, cause the head mounted display device to: If there are two or more people within the above specified range, it will check the actions of the user or people, A head-mounted display device that identifies a person based on at least one of the user's actions or the people's actions.

3. In paragraph 2, The above instructions, when individually or collectively executed by the at least one processor, cause the head mounted display device to: A head-mounted display device that measures a second distance to a specified person based on at least one sensor, controls a focal length of the lenticular lens or the lens array of the 3D display based on the second distance, and displays an image on the 3D display.

4. In paragraph 2, The above instructions, when individually or collectively executed by the at least one processor, cause the head mounted display device to: Tracking the user's gaze based on at least one sensor, A head-mounted display device that identifies a person the user is looking at based on the tracked gaze.

5. In paragraph 2, The above instructions, when individually or collectively executed by the at least one processor, cause the head mounted display device to: Verify the user's conversation based on at least one sensor, A head-mounted display device that identifies a person based on a conversation with the user.

6. In paragraph 2, The above instructions, when individually or collectively executed by the at least one processor, cause the head mounted display device to: Based on at least one sensor, the behavior of the people is confirmed, Based on the actions of the people identified above: Identifying the person based on the priority in response to the actions of the person; or Based on at least one sensor, identify a person among the people looking at the head mounted display device, or Based on at least one sensor, identify a person among the people who is talking to the user of the head-mounted display device, or A head mounted display device that identifies a person close to a user of the head mounted display device among the people based on at least one of the sensors.

7. In paragraph 1, The above lenticular lens or the above lens array At least one lens having a variable focal length; and comprising at least one electrode corresponding to at least one lens, The above instructions, when individually or collectively executed by the at least one processor, cause the head mounted display device to: Calculate the focal length to control the viewing distance corresponding to the first distance, A head-mounted display device that applies a voltage corresponding to the calculated focal distance to at least one electrode.

8. In a display control method of a head mounted display device, An action to detect wearing of a head mounted display device based on at least one sensor; An action of detecting a person in the user's surroundings based on at least one sensor; An action that determines whether a person exists within a specified range; An action to determine whether there is one person within the specified range; A method comprising: measuring a first distance to a person based on at least one sensor, if a person exists within the specified range, controlling a focal length of a lenticular lens or lens array of a 3D display based on the first distance, and displaying an image on the 3D display.

9. In paragraph 8, Actions to verify the user's actions or the actions of people when there are two or more people within a specified range; and A method comprising an action of identifying a person based on at least one of the user's actions or the actions of the people.

10. In paragraph 9, A method comprising: measuring a second distance to a specified person based on at least one sensor; controlling a focal length of the lenticular lens or the lens array of the 3D display based on the second distance; and displaying an image on the 3D display.

11. In paragraph 9, An operation to track the user's gaze based on at least one sensor; and A method including an action to identify the person the user is looking at based on the tracked gaze.

12. In Paragraph 9, An operation to confirm the user's conversation based on at least one sensor; and A method including an action to identify a person based on a conversation with the above-mentioned user.

13. In Paragraph 9, Based on at least one sensor, the operation includes verifying the behavior of the people, and Based on the behavior of the people identified above: An action that identifies the said person based on a priority corresponding to the actions of the said people; An action of identifying a person among the people looking at the head-mounted display device based on at least one sensor; An action of identifying a person among the people who converses with the user of the head-mounted display device based on at least one of the above sensors; or A method comprising at least one action of identifying a person close to the user of the head-mounted display device among the people based on at least one sensor.

14. In Paragraph 8, The above lenticular lens or the above lens array is At least one lens with a changing focal length; and A method comprising at least one electrode corresponding to at least one lens.

15. In Paragraph 14, An operation to calculate the focal length for controlling the viewing distance corresponding to the first distance; and A method comprising the operation of applying a voltage corresponding to the above-calculated focal length to at least one electrode.

Images