Electronic device, method, and computer-readable storage medium for changing screen on basis of switching of virtual space

Abstract

According to an embodiment, a wearable device comprises: a display; a camera; at least one sensor; a memory for storing instructions; and at least one processor. The at least one processor is configured to, when the instructions are executed, display a first screen related to a first virtual space while the display mode of the display is in a first display mode. The at least one processor is configured to, when the instructions are executed, identify information about an external environment in which the wearable device is located. The at least one processor is configured to, when the instructions are executed, identify that the display mode of the display is changed from the first display mode to a second display mode. The at least one processor is configured to identify a second screen related to a second virtual space according to the second display mode. The at least one processor is configured to, when the instructions are executed, display a third screen changed from the second screen, while the display mode of the display is in the second display mode.

Description

Electronic device, method, and computer-readable storage medium for changing a screen based on a transition of a virtual space

[0001] The following descriptions relate to an electronic device, a method, and a computer-readable storage medium for changing a screen based on a transition in a virtual space.

[0002] To provide an enhanced user experience, electronic devices are being developed that provide augmented reality (AR) and / or virtual reality (VR) services that display computer-generated information in conjunction with external objects in the real world. The electronic devices may be wearable devices that can be worn by a user. For example, the electronic devices may be AR glasses and / or head-mounted devices (HMDs).

[0003] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art related to the present disclosure.

[0004] According to one embodiment, a wearable device may include a display, a camera, at least one sensor, a memory storing instructions and including one or more storage media, and at least one processor including a processing circuit. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display a first screen regarding a first virtual space through the display while a display mode of the display is a first display mode. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify information regarding an external environment in which the wearable device is located while at least one screen including the first screen regarding the first virtual space is displayed. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify that the display mode of the display is changed from the first display mode to a second display mode after the information about the external environment is identified. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify a second screen relating to a second virtual space according to the second display mode based on identifying that the display mode of the display is changed from the first display mode to the second display mode.The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display a third screen changed from the second screen through the display while the display mode of the display is the second display mode, based on the first area within the second screen identified based on the information about the external environment and the gaze of the user of the wearable device.

[0005] According to one embodiment, a method of a wearable device may include an operation of displaying a first screen regarding a first virtual space through the display while the display mode of the display is a first display mode. The method may include an operation of identifying information regarding an external environment in which the wearable device is located while at least one screen including the first screen regarding the first virtual space is displayed. The method may include an operation of identifying, after the information regarding the external environment is identified, that the display mode of the display is changed from the first display mode to a second display mode. The method may include an operation of identifying a second screen regarding a second virtual space according to the second display mode based on identifying that the display mode of the display is changed from the first display mode to the second display mode. The method may include an operation of displaying a third screen changed from the second screen through the display while the display mode of the display is the second display mode, based on the information about the external environment and the first area within the second screen identified according to the gaze of the user of the wearable device.

[0006] According to one embodiment, a non-transitory computer readable storage medium may store one or more programs. The one or more programs may include instructions that, when executed by at least one processor of a wearable device having a display, a camera, and at least one sensor, cause the wearable device to display a first screen regarding a first virtual space through the display while the display mode of the display is a first display mode. The one or more programs may include instructions that cause the wearable device to identify information about an external environment while at least one screen including the first screen regarding the first virtual space is displayed. The one or more programs may include instructions that, after the information about the external environment is identified, cause the wearable device to identify that the display mode of the display is changed from the first display mode to a second display mode. The one or more programs may include instructions that cause the wearable device to identify a second screen related to a second virtual space according to the second display mode based on identifying that the display mode of the display has changed from the first display mode to the second display mode. The one or more programs may include instructions that cause the wearable device to display, through the display, a third screen changed from the second screen based on a first area within the second screen identified according to the information about the external environment and the gaze of the user of the wearable device while the display mode of the display is the second display mode.

[0007] FIG. 1 is a block diagram of an electronic device within a network environment according to one embodiment.

[0008] FIGS. 2A and 2B illustrate an example of a perspective view of a wearable device according to one embodiment.

[0009] FIGS. 3A and 3B illustrate an example of an appearance of a wearable device according to one embodiment.

[0010] FIG. 4 illustrates an example block diagram of a wearable device according to one embodiment.

[0011] FIG. 5 illustrates examples of virtual spaces that change depending on the display mode of the display, according to one embodiment.

[0012] FIG. 6A illustrates a flowchart of the operation of a wearable device according to one embodiment.

[0013] FIG. 6b illustrates a flowchart of the operation of a wearable device according to one embodiment.

[0014] FIG. 7A illustrates a flowchart of the operation of a wearable device according to one embodiment.

[0015] FIG. 7b illustrates an example of an operation for identifying a location where a light source is placed within a space where a wearable device is located, according to one embodiment.

[0016] FIG. 8 illustrates a flowchart of the operation of a wearable device according to one embodiment.

[0017] FIG. 9 illustrates a flowchart of the operation of a wearable device according to one embodiment.

[0018] FIG. 10 illustrates an example of operation of a wearable device according to one embodiment.

[0019] FIG. 11 illustrates an example of operation of a wearable device according to one embodiment.

[0020] FIG. 12 illustrates an example of operation of a wearable device according to one embodiment.

[0021] FIG. 13 illustrates an example of operation of a wearable device according to one embodiment.

[0022] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0023] FIG. 1 is a block diagram of an electronic device within a network environment according to one embodiment.

[0024] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0025] The processor (120) may control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) by executing, for example, software (e.g., a program (140)), and may perform various data processing or calculations. According to one embodiment, as at least a part of the data processing or calculation, the processor (120) may store a command or data received from another component (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the command or data stored in the volatile memory (132), and store the resulting data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or a secondary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together therewith. For example, if the electronic device (101) includes a main processor (121) and a secondary processor (123), the secondary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a specified function. The secondary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0026] The auxiliary processor (123) may control at least a part of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0027] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0028] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0029] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0030] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. According to one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0031] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0032] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0033] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0034] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0035] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0036] A haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0037] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0038] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0039] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0040] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0041] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) may support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0042] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. According to some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0043] In one embodiment, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0044] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0045] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service by itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that have received the request may execute at least a part of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101).

[0046] The electronic device (101) may process the result as is or additionally and provide it as at least part of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199). The electronic device (101) may be applied to intelligent services (e.g., smart homes, smart cities, smart cars, or healthcare) based on 5G communication technology and IoT-related technology.

[0047] FIGS. 2A and 2B illustrate an example of a perspective view of a wearable device according to one embodiment.

[0048] According to one embodiment, the wearable device (200) may have the form of glasses that are wearable on a body part of a user (e.g., head). The wearable device (200) of FIGS. 2A and 2B may be an example of the electronic device (101) of FIG. 1. The wearable device (200) may include a head-mounted display (HMD). For example, the housing of the wearable device (200) may include a flexible material, such as rubber and / or silicone, that is configured to fit closely to a portion of the user's head (e.g., a portion of the face surrounding both eyes). For example, the housing of the wearable device (200) may include one or more straps that are capable of being twined around the user's head, and / or one or more temples that are detachably attachable to the ears of the head.

[0049] Referring to FIG. 2A, according to one embodiment, a wearable device (200) may include at least one display (250) and a frame (200) supporting at least one display (250).

[0050] According to one embodiment, the wearable device (200) can be worn on a part of a user's body. The wearable device (200) can provide augmented reality (AR), virtual reality (VR), or mixed reality (MR) that combines augmented reality and virtual reality to the user wearing the wearable device (200). For example, the wearable device (200) can display a virtual reality image provided from at least one optical device (282, 284) of FIG. 2B on at least one display (250) in response to a user's designated gesture acquired through the motion recognition cameras (260-2, 263) of FIG. 2B.

[0051] According to one embodiment, at least one display (250) may provide visual information to a user. For example, at least one display (250) may include a transparent or translucent lens. At least one display (250) may include a first display (250-1) and / or a second display (250-2) spaced apart from the first display (250-1). For example, the first display (250-1) and the second display (250-2) may be positioned at positions corresponding to the user's left and right eyes, respectively.

[0052] Referring to FIG. 2B, at least one display (250) can provide the user with visual information transmitted from external light and other visual information distinct from the visual information through a lens included in the at least one display (250). The lens can be formed based on at least one of a Fresnel lens, a pancake lens, or a multi-channel lens. For example, the at least one display (250) can include a first surface (231) and a second surface (232) opposite to the first surface (231). A display area can be formed on the second surface (232) of the at least one display (250). When the user wears the wearable device (200), external light can be transmitted to the user by being incident on the first surface (231) and transmitted through the second surface (232). As another example, at least one display (250) can display an augmented reality image combined with a virtual reality image provided from at least one optical device (282, 284) on a real screen transmitted through external light, in a display area formed on the second surface (232).

[0053] In one embodiment, at least one display (250) may include at least one waveguide (233, 234) that diffracts light emitted from at least one optical device (282, 284) and transmits the diffracted light to a user. The at least one waveguide (233, 234) may be formed based on at least one of glass, plastic, or polymer. A nano-pattern may be formed on at least a portion of the exterior or interior of the at least one waveguide (233, 234). The nano-pattern may be formed based on a grating structure having a polygonal and / or curved shape. Light incident on one end of the at least one waveguide (233, 234) may be propagated to the other end of the at least one waveguide (233, 234) by the nano-pattern. At least one waveguide (233, 234) may include at least one diffractive element (e.g., a diffractive optical element (DOE), a holographic optical element (HOE)) and at least one reflective element (e.g., a reflective mirror). For example, at least one waveguide (233, 234) may be arranged within the wearable device (200) to guide a screen displayed by at least one display (250) to the user's eyes. For example, the screen may be transmitted to the user's eyes based on total internal reflection (TIR) ​​occurring within the at least one waveguide (233, 234).

[0054] The wearable device (200) can analyze an object included in a real image collected through a shooting camera (260-4), combine a virtual object corresponding to an object to be provided with augmented reality among the analyzed objects, and display the virtual object on at least one display (250). The virtual object can include at least one of text and an image regarding various information related to the object included in the real image. The wearable device (200) can analyze the object based on a multi-camera such as a stereo camera. For the object analysis, the wearable device (200) can perform spatial recognition (e.g., simultaneous localization and mapping (SLAM)) using a multi-camera and / or time-of-flight (ToF). A user wearing the wearable device (200) can view an image displayed on at least one display (250).

[0055] According to one embodiment, the frame (200) may be formed as a physical structure that allows the wearable device (200) to be worn on the user's body. According to one embodiment, the frame (200) may be configured so that, when the user wears the wearable device (200), the first display (250-1) and the second display (250-2) can be positioned corresponding to the user's left and right eyes. The frame (200) may support at least one display (250). For example, the frame (200) may support the first display (250-1) and the second display (250-2) to be positioned corresponding to the user's left and right eyes.

[0056] Referring to FIG. 2A, the frame (200) may include a region (220) that is at least partially in contact with a part of the user's body when the user wears the wearable device (200). For example, the region (220) of the frame (200) that is in contact with a part of the user's body may include a region that is in contact with a part of the user's nose, a part of the user's ear, and a part of the side of the user's face that the wearable device (200) makes contact with. According to one embodiment, the frame (200) may include a nose pad (210) that is in contact with a part of the user's body. When the wearable device (200) is worn by the user, the nose pad (210) may be in contact with a part of the user's nose. The frame (200) may include a first temple (204) and a second temple (205) that are in contact with another part of the user's body that is distinct from the part of the user's body.

[0057] For example, the frame (200) may include a first rim (201) that surrounds at least a portion of the first display (250-1), a second rim (202) that surrounds at least a portion of the second display (250-2), a bridge (203) that is disposed between the first rim (201) and the second rim (202), a first pad (211) that is disposed along a portion of the edge of the first rim (201) from one end of the bridge (203), a second pad (212) that is disposed along a portion of the edge of the second rim (202) from the other end of the bridge (203), a first temple (204) that extends from the first rim (201) and is fixed to a portion of the wearer's ear, and a second temple (205) that extends from the second rim (202) and is fixed to a portion of the ear opposite the ear. The first pad (211) and the second pad (212) may be in contact with a portion of the user's nose, and the first temple (204) and the second temple (205) may be in contact with a portion of the user's face and a portion of the user's ear. The temples (204, 205) may be rotatably connected to the rim through the hinge units (206, 207) of FIG. 2B. The first temple (204) may be rotatably connected to the first rim (201) through the first hinge unit (206) disposed between the first rim (201) and the first temple (204). The second temple (205) may be rotatably connected to the second rim (202) through the second hinge unit (207) disposed between the second rim (202) and the second temple (205). According to one embodiment, the wearable device (200) can identify an external object (e.g., a user's fingertip) touching the frame (200) and / or a gesture performed by the external object by using a touch sensor, a grip sensor, and / or a proximity sensor formed on at least a portion of a surface of the frame (200).

[0058] According to one embodiment, the wearable device (200) may include hardwares that perform various functions (e.g., hardwares to be described later based on the block diagram of FIG. 4). For example, the hardwares may include a battery module (270), an antenna module (275), at least one optical device (282, 284), speakers (e.g., speakers 255-1, 255-2), a microphone (e.g., microphones 265-1, 265-2, 265-3), a light-emitting module (not shown), and / or a printed circuit board (PCB) (290) (e.g., a printed circuit board). The various hardwares may be arranged within the frame (200).

[0059] According to one embodiment, the microphones (e.g., microphones 265-1, 265-2, 265-3) of the wearable device (200) may be disposed on at least a portion of the frame (200) to acquire sound signals. A first microphone (265-1) disposed on the bridge (203), a second microphone (265-2) disposed on the second rim (202), and a third microphone (265-3) disposed on the first rim (201) are illustrated in FIG. 2B , but the number and arrangement of the microphones (265) are not limited to the embodiment of FIG. 2B . When the number of microphones (265) included in the wearable device (200) is two or more, the wearable device (200) may identify the direction of the sound signal by using a plurality of microphones disposed on different portions of the frame (200).

[0060] In one embodiment, at least one optical device (282, 284) can project a virtual object onto at least one display (250) to provide various image information to a user. For example, at least one optical device (282, 284) can be a projector. At least one optical device (282, 284) can be disposed adjacent to at least one display (250) or can be included within at least one display (250) as a part of at least one display (250). In one embodiment, the wearable device (200) can include a first optical device (282) corresponding to a first display (250-1) and a second optical device (284) corresponding to a second display (250-2). For example, at least one optical device (282, 284) may include a first optical device (282) disposed at an edge of a first display (250-1) and a second optical device (284) disposed at an edge of a second display (250-2). The first optical device (282) may transmit light to a first waveguide (233) disposed on the first display (250-1), and the second optical device (284) may transmit light to a second waveguide (234) disposed on the second display (250-2).

[0061] In one embodiment, the camera (260) may include a recording camera (260-4), an eye tracking camera (ET CAM) (260-1), and / or a motion recognition camera (260-2, 260-3). The recording camera (260-4), the eye tracking camera (260-1), and the motion recognition cameras (260-2, 260-3) may be positioned at different locations on the frame (200) and may perform different functions. The eye tracking camera (260-1) may output data indicating the position or gaze of the eyes of a user wearing the wearable device (200). For example, the wearable device (200) may detect the gaze from an image including the user's pupils obtained through the eye tracking camera (260-1). The wearable device (200) can identify an object (e.g., a real object and / or a virtual object) focused on by the user using the user's gaze acquired through the gaze tracking camera (260-1). The wearable device (200) that has identified the focused object can execute a function (e.g., gaze interaction) for interaction between the user and the focused object. The wearable device (200) can express a part corresponding to the eye of an avatar representing the user in a virtual space using the user's gaze acquired through the gaze tracking camera (260-1). The wearable device (200) can render an image (or screen) displayed on at least one display (250) based on the position of the user's eyes. For example, the visual quality of a first region related to the gaze within the image and the visual quality (e.g., resolution, brightness, saturation, grayscale, PPI) of a second region distinguished from the first region may be different from each other. The wearable device (200) can obtain an image having visual quality of a first area matching the user's gaze and visual quality of a second area using foveated rendering.For example, if the wearable device (200) supports an iris recognition function, user authentication can be performed based on iris information acquired using the gaze tracking camera (260-1). An example in which the gaze tracking camera (260-1) is positioned toward the user's right eye is illustrated in FIG. 2B, but the embodiment is not limited thereto, and the gaze tracking camera (260-1) may be positioned solely toward the user's left eye, or toward both eyes.

[0062] In one embodiment, the capturing camera (260-4) can capture an actual image or background to be aligned with a virtual image to implement augmented reality or mixed reality content. The capturing camera (260-4) can be used to acquire a high-resolution image based on HR (high resolution) or PV (photo video). The capturing camera (260-4) can capture an image of a specific object existing at a location viewed by the user and provide the image to at least one display (250). The at least one display (250) can display a single image in which information about an actual image or background including the image of the specific object acquired using the capturing camera (260-4) is superimposed on a virtual image provided through at least one optical device (282, 284). The wearable device (200) can compensate for depth information (e.g., the distance between the wearable device (200) and an external object acquired through a depth sensor) using the image acquired through the capturing camera (260-4). The wearable device (200) can perform object recognition through an image acquired using a photographing camera (260-4). The wearable device (200) can perform a function of focusing on an object (or subject) in an image (e.g., auto focus) and / or an optical image stabilization (OIS) function (e.g., anti-shake function) using the photographing camera (260-4). The wearable device (200) can perform a pass-through function to display an image acquired through the photographing camera (260-4) by overlapping at least a portion of a screen representing a virtual space on at least one display (250). In one embodiment, the photographing camera (260-4) can be disposed on a bridge (203) disposed between a first rim (201) and a second rim (202).

[0063] The gaze tracking camera (260-1) can implement more realistic augmented reality by tracking the gaze of a user wearing the wearable device (200) and matching the user's gaze with visual information provided to at least one display (250). For example, when the wearable device (200) looks straight ahead, the wearable device (200) can naturally display environmental information related to the user's front at a location where the user is located on at least one display (250). The gaze tracking camera (260-1) can be configured to capture an image of the user's pupil to determine the user's gaze. For example, the gaze tracking camera (260-1) can receive gaze detection light reflected from the user's pupil and track the user's gaze based on the position and movement of the received gaze detection light. In one embodiment, the gaze tracking camera (260-1) can be positioned at positions corresponding to the user's left and right eyes. For example, the gaze tracking camera (260-1) may be positioned within the first rim (201) and / or the second rim (202) to face the direction in which the user wearing the wearable device (200) is positioned.

[0064] The gesture recognition camera (260-2, 260-3) can recognize the movement of the user's entire body, such as the user's torso, hand, or face, or a part of the body, and thereby provide a specific event on a screen provided on at least one display (250). The gesture recognition camera (260-2, 260-3) can recognize the user's gesture (gesture recognition), obtain a signal corresponding to the gesture, and provide a display corresponding to the signal on at least one display (250). The processor can identify the signal corresponding to the gesture, and perform a designated function based on the identification. The gesture recognition camera (260-2, 260-3) can be used to perform simultaneous localization and mapping (SLAM) for 6 degrees of freedom pose (6 dof pose) and / or a spatial recognition function using a depth map. The processor may perform gesture recognition and / or object tracking functions using the motion recognition cameras (260-2, 260-3). In one embodiment, the motion recognition cameras (260-2, 260-3) may be positioned on the first rim (201) and / or the second rim (202).

[0065] The camera (260) included in the wearable device (200) is not limited to the above-described gaze tracking camera (260-1) and motion recognition cameras (260-2, 260-3). For example, the wearable device (200) can identify an external object included in the FoV using a camera positioned toward the user's FoV. The wearable device (200) can identify an external object based on a sensor for identifying the distance between the wearable device (200) and the external object, such as a depth sensor and / or a time of flight (ToF) sensor. The camera (260) positioned toward the FoV can support an autofocus function and / or an optical image stabilization (OIS) function. For example, the wearable device (200) may include a camera (260) (e.g., a face tracking (FT) camera) positioned toward the face to obtain an image including the face of a user wearing the wearable device (200).

[0066] Although not shown, in one embodiment, the wearable device (200) may further include a light source (e.g., an LED) that emits light toward a subject (e.g., a user's eyes, face, and / or an external object within the FoV) being photographed using the camera (260). The light source may include an LED having an infrared wavelength. The light source may be disposed on at least one of the frame (200) and the hinge units (206, 207).

[0067] According to one embodiment, the battery module (270) may supply power to electronic components of the wearable device (200). In one embodiment, the battery module (270) may be disposed within the first temple (204) and / or the second temple (205). For example, the battery module (270) may be a plurality of battery modules (270). The plurality of battery modules (270) may be disposed within each of the first temple (204) and the second temple (205). In one embodiment, the battery module (270) may be disposed at an end of the first temple (204) and / or the second temple (205).

[0068] The antenna module (275) can transmit signals or power to the outside of the wearable device (200), or receive signals or power from the outside. In one embodiment, the antenna module (275) can be positioned within the first temple (204) and / or the second temple (205). For example, the antenna module (275) can be positioned close to one surface of the first temple (204) and / or the second temple (205).

[0069] The speaker (255) can output an audio signal to the outside of the wearable device (200). The audio output module may be referred to as a speaker. In one embodiment, the speaker (255) may be positioned within the first temple (204) and / or the second temple (205) so as to be positioned adjacent to the ear of a user wearing the wearable device (200). For example, the speaker (255) may include a second speaker (255-2) positioned within the first temple (204) and thus adjacent to the user's left ear, and a first speaker (255-1) positioned within the second temple (205) and thus adjacent to the user's right ear.

[0070] The light-emitting module (not shown) may include at least one light-emitting element. The light-emitting module may emit light of a color corresponding to a specific state or emit light with an action corresponding to a specific state in order to visually provide information regarding a specific state of the wearable device (200) to the user. For example, when the wearable device (200) requires charging, it may emit red light at a regular cycle. In one embodiment, the light-emitting module may be disposed on the first rim (201) and / or the second rim (202).

[0071] Referring to FIG. 2B, according to one embodiment, a wearable device (200) may include a printed circuit board (PCB) (290). The PCB (290) may be included in at least one of the first temple (204) or the second temple (205). The PCB (290) may include an interposer disposed between at least two sub-PCBs. One or more hardwares included in the wearable device (200) (e.g., hardwares illustrated by different blocks in FIG. 4) may be disposed on the PCB (290). The wearable device (200) may include a flexible PCB (FPCB) for interconnecting the hardwares.

[0072] According to one embodiment, a wearable device (200) may include at least one of a gyro sensor, a gravity sensor, and / or an acceleration sensor for detecting a posture of the wearable device (200) and / or a posture of a body part (e.g., a head) of a user wearing the wearable device (200). Each of the gravity sensor and the acceleration sensor may measure gravitational acceleration and / or acceleration based on mutually perpendicular designated three-dimensional axes (e.g., the x-axis, the y-axis, and the z-axis). The gyro sensor may measure an angular velocity of each of the designated three-dimensional axes (e.g., the x-axis, the y-axis, and the z-axis). At least one of the gravity sensor, the acceleration sensor, and the gyro sensor may be referred to as an inertial measurement unit (IMU). According to one embodiment, the wearable device (200) may identify a user's motion and / or gesture performed to execute or terminate a specific function of the wearable device (200) based on the IMU.

[0073] Figures 3a and 3b illustrate an example of an exterior appearance of a wearable device according to one embodiment. The wearable device (300) of Figures 3a and 3b may be an example of the electronic device (101) of Figure 1 and the wearable device (200) of Figures 2a and 2b. An example of an exterior appearance of a first side (310) of a housing of the wearable device (200) according to one embodiment is illustrated in Figure 3a, and an example of an exterior appearance of a second side (320) opposite to the first side (310) may be illustrated in Figure 3b.

[0074] Referring to FIG. 3A, according to one embodiment, a first surface (310) of a wearable device (200) may have a form attachable to a body part of a user (e.g., the face of the user). Although not shown, the wearable device (200) may further include a strap for fixing to a body part of a user, and / or one or more temples (e.g., the first temple (204) and / or the second temple (205) of FIGS. 2A and 2B). A first display (250-1) for outputting an image to a left eye among the user's two eyes, and a second display (250-2) for outputting an image to a right eye among the user's two eyes may be disposed on the first surface (310). The wearable device (200) is formed on the first surface (310) and may further include a rubber or silicone packing to prevent interference by light (e.g., ambient light) different from the light emitted from the first display (250-1) and the second display (250-2).

[0075] According to one embodiment, the wearable device (200) may include cameras (260-1) for photographing and / or tracking both eyes of the user adjacent to each of the first display (250-1) and the second display (250-2). The cameras (260-1) may be referred to as the gaze tracking camera (260-1) of FIG. 2B. According to one embodiment, the wearable device (200) may include cameras (260-5, 260-6) for photographing and / or recognizing the face of the user. The cameras (260-5, 260-6) may be referred to as FT cameras. The wearable device (200) may control an avatar representing the user in a virtual space based on the motion of the user's face identified using the cameras (260-5, 260-6). For example, the wearable device (200) may change the texture and / or shape of a portion of an avatar (e.g., a portion of an avatar representing a human face) using information obtained by cameras (260-5, 260-6) (e.g., FT cameras) and representing the facial expression of a user wearing the wearable device (200).

[0076] Referring to FIG. 3B, a camera (e.g., cameras 260-7, 260-8, 260-9, 260-10, 260-11, 260-12)) and / or a sensor (e.g., a depth sensor 330) may be disposed on a second surface (320) opposite to the first surface (310) of FIG. 3A to obtain information related to the external environment of the wearable device (200). For example, the cameras (260-7, 260-8, 260-9, 260-10) may be disposed on the second surface (320) to recognize external objects. Cameras (260-7, 260-8, 260-9, 260-10) may be referenced to the motion recognition cameras (260-2, 260-3) of FIG. 2B.

[0077] For example, using cameras (260-11, 260-12), the wearable device (200) can obtain images and / or videos to be transmitted to each of the user's eyes. The camera (260-11) can be placed on the second face (320) of the wearable device (200) to obtain an image to be displayed through the second display (250-2) corresponding to the right eye among the two eyes. The camera (260-12) can be placed on the second face (320) of the wearable device (200) to obtain an image to be displayed through the first display (250-1) corresponding to the left eye among the two eyes. The cameras (260-11, 260-12) can be referred to as the shooting camera (260-4) of FIG. 2B.

[0078] According to one embodiment, the wearable device (200) may include a depth sensor (330) disposed on the second face (320) to identify a distance between the wearable device (200) and an external object. Using the depth sensor (330), the wearable device (200) may obtain spatial information (e.g., a depth map) for at least a portion of the FoV of a user wearing the wearable device (200). Although not illustrated, a microphone may be disposed on the second face (320) of the wearable device (200) to obtain sound output from an external object. The number of microphones may be one or more, depending on the embodiment.

[0079] FIG. 4 illustrates an example block diagram of a wearable device according to one embodiment. Referring to FIG. 4, a wearable device (200) according to one embodiment may include at least one of a processor (410), a memory (415), a display (420), a camera (425), a sensor (430), or a communication circuit (435). The processor (410), the memory (415), the display (420), the camera (425), the sensor (430), and the communication circuit (435) may be electrically and / or operably coupled with each other by an electronic component such as a communication bus (402). The type and / or number of hardware components included in the wearable device (200) are not limited to those illustrated in FIG. 4. For example, the wearable device (200) may include only some of the hardware components illustrated in FIG. 4. The elements within the memory described below (e.g., layers and / or modules) may be logically distinct, but are not limited thereto.

[0080] According to one embodiment, the processor (410) of the wearable device (200) may include a hardware component for processing data based on one or more instructions. The hardware component for processing data may include, for example, an arithmetic and logic unit (ALU), a field programmable gate array (FPGA), and / or a central processing unit (CPU). The number of processors (410) may be one or more. For example, the processor (410) may have a multi-core processor structure such as a dual core, a quad core, or a hexa core.

[0081] The memory (415) of the wearable device (200) according to one embodiment may include a hardware component for storing data and / or instructions input and / or output to the processor (410). The memory (415) may include, for example, a volatile memory such as a random-access memory (RAM) and / or a non-volatile memory such as a read-only memory (ROM). The volatile memory may include, for example, at least one of a dynamic RAM (DRAM), a static RAM (SRAM), a cache RAM, and a pseudo SRAM (PSRAM). The non-volatile memory may include, for example, at least one of a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), a flash memory, a hard disk, a compact disc, and an embedded multi media card (eMMC).

[0082] In one embodiment, the display (420) of the wearable device (200) can output visualized information to a user of the wearable device (200). For example, the display (420) can be controlled by a processor (410) including a circuit such as a graphic processing unit (GPU) to output visualized information to the user. The display (420) can include a flat panel display (FPD) and / or electronic paper. The FPD can include a liquid crystal display (LCD), a plasma display panel (PDP), and / or one or more light emitting diodes (LEDs). The LEDs can include organic LEDs (OLEDs).

[0083] In one embodiment, the camera (425) of the wearable device (200) may include one or more optical sensors (e.g., a charged coupled device (CCD) sensor, a complementary metal oxide semiconductor (CMOS) sensor) that generate electrical signals representing the color and / or brightness of light. The plurality of optical sensors included in the camera (425) may be arranged in the form of a two-dimensional array. The camera (425) may acquire electrical signals of each of the plurality of optical sensors substantially simultaneously to generate two-dimensional frame data corresponding to light reaching the optical sensors of the two-dimensional array. For example, photographic data captured using the camera (425) may mean one (a) two-dimensional frame data acquired from the camera (425). For example, video data captured using the camera (425) may mean a sequence of a plurality of two-dimensional frame data acquired from the camera (425) according to a frame rate. The camera (425) may further include a flash light that is positioned toward the direction in which the camera (425) receives light and outputs light toward the direction.

[0084] According to one embodiment, the wearable device (200) may include a plurality of cameras, for example, cameras (425), arranged in different directions. A first camera among the plurality of cameras may be referred to as a motion recognition camera (e.g., motion recognition cameras 260-2 and 260-3 of FIG. 2B ), and a second camera may be referred to as a gaze tracking camera (e.g., gaze tracking camera 260-1 of FIG. 2B ). The wearable device (200) may identify a position, shape, and / or gesture of a hand using an image acquired using the first camera. The wearable device (200) may identify a direction of a gaze of a user wearing the wearable device (200) using an image acquired using the second camera. For example, the direction in which the first camera faces may be opposite to the direction in which the second camera faces.

[0085] According to one embodiment, a sensor (430) of a wearable device (200) may generate electrical information that may be processed by a processor (410) and / or a memory (415) of the wearable device (200) from non-electronic information related to the wearable device (200). The information may be referred to as sensor data. The sensor (430) may include a global positioning system (GPS) sensor, an image sensor, an ambient light sensor, and / or a time-of-flight (ToF) sensor for detecting a geographic location of the wearable device (200), and an inertial measurement unit (IMU) for detecting a physical motion of the wearable device (200).

[0086] In one embodiment, the communication circuit (435) of the wearable device (200) may include hardware components for supporting transmission and / or reception of electrical signals between the wearable device (200) and an external electronic device. The communication circuit (435) may include, for example, at least one of a modem (MODEM), an antenna, and an optical / electronic (O / E) converter. The communication circuit (435) may support transmission and / or reception of electrical signals based on various types of protocols, such as Ethernet, a local area network (LAN), a wide area network (WAN), wireless fidelity (WiFi), Bluetooth, Bluetooth low energy (BLE), ZigBee, long term evolution (LTE), 5G NR (new radio), and / or 6G.

[0087] According to one embodiment, one or more instructions (or commands) representing operations and / or actions to be performed on data by the processor (410) of the wearable device (200) may be stored in the memory (415) of the wearable device (200). A set of one or more instructions may be referred to as firmware, an operating system, a process, a routine, a sub-routine, and / or an application. For example, the wearable device (200) and / or the processor (410) may perform at least one of the operations according to the embodiments described below when a set of a plurality of instructions distributed in the form of an operating system, firmware, a driver, and / or an application is executed. Hereinafter, the fact that an application is installed in a wearable device (200) may mean that one or more instructions provided in the form of an application are stored in a memory (415), and that the one or more applications are stored in a format executable by the processor (410) (e.g., a file having an extension specified by the operating system of the wearable device (200)). As an example, an application may include a program and / or a library related to a service provided to a user.

[0088] Referring to FIG. 4, programs installed in the wearable device (200) may be classified into any one of different layers, including an application layer (440), a framework layer (450), and / or a hardware abstraction layer (HAL) (480), based on the target. For example, programs (e.g., modules or drivers) designed to target the hardware of the wearable device (200) (e.g., the display (420), the camera (420), and / or the sensor (430)) may be classified within the hardware abstraction layer (480). The framework layer (450) may be referred to as an XR framework layer in that it includes one or more programs for providing an XR (extended reality) service. For example, FIG. 4 illustrates layers being divided within the memory (415), the layers may be logically divided. However, the present invention is not limited thereto. Depending on the embodiment, the layers may also be stored in a designated area within the memory (415).

[0089] For example, within the framework layer (450), programs designed to target at least one of the hardware abstraction layer (480) and / or the application layer (440) (e.g., a position tracker (471), a space recognizer (472), a gesture tracker (473), and / or an eye tracker (474), a face tracker (475)) may be classified. Programs classified within the framework layer (450) may provide an executable API (application programming interface) based on other programs.

[0090] For example, within the application layer (440), programs designed to target users controlling wearable devices (200) may be classified. Examples of programs classified into the application layer (440) include, but are not limited to, an XR (extended reality) system UI (user interface) and / or an XR application (442). For example, programs (e.g., software applications) classified into the application layer (440) may call an API (application programming interface) to cause execution of functions supported by programs classified into the framework layer (450).

[0091] For example, the wearable device (200) may display one or more visual objects on the display (420) for performing interaction with a user for using a virtual space based on the execution of the XR system UI (441). A visual object may refer to an object that can be deployed within a screen for transmitting and / or interacting with information, such as text, an image, an icon, a video, a button, a checkbox, a radio button, a text box, a slider, and / or a table. A visual object may be referred to as a visual guide, a virtual object, a visual element, a UI element, a view object, and / or a view element. The wearable device (200) may provide a service for controlling functions available within a virtual space to the user based on the execution of the XR system UI (441).

[0092] Referring to FIG. 4, a lightweight renderer (443) and / or an XR plug-in (444) are illustrated as being included within the XR system UI (441), but are not limited thereto. For example, the XR system UI (441) may cause execution of functions supported by the lightweight renderer (443) and / or the XR plug-in (444) included within the framework layer (450).

[0093] For example, the wearable device (200) may obtain resources (e.g., APIs, system processes, and / or libraries) used to define, create, and / or execute a rendering pipeline that allows partial changes based on the execution of a lightweight renderer (442). The lightweight renderer (443) may be referred to as a lightweight render pipeline in terms of defining a rendering pipeline that allows partial changes. The lightweight renderer (443) may include a renderer built prior to the execution of a software application (e.g., a prebuilt renderer). For example, the wearable device (200) may obtain resources (e.g., APIs, system processes, and / or libraries) used to define, create, and / or execute an entire rendering pipeline based on the execution of an XR plug-in (444). The XR plugin (444) may be referred to as an open XR native client from the perspective of defining (or configuring) the entire rendering pipeline.

[0094] For example, the wearable device (200) may display a screen representing at least a portion of a virtual space on the display (420) based on the execution of the XR application (442). The XR plug-in (444-1) included in the XR application (442) may be referenced by the XR plug-in (444) of the XR system UI (441). Descriptions of the XR plug-in (444-1) that overlap with the description of the XR plug-in (444) may be omitted. The wearable device (200) may cause the execution of the virtual space manager (451) based on the execution of the XR application (442).

[0095] According to one embodiment, the wearable device (200) may provide a virtual space service based on the execution of the virtual space manager (451). For example, the virtual space manager (451) may include a platform (e.g., an Android platform) for supporting the virtual space service. The wearable device (200) may display the posture of a virtual object representing the user's posture rendered using data acquired through the sensor (430) based on the execution of the virtual space manager (451) on the display. The virtual space manager (451) may be referred to as a composition presentation manager (CPM).

[0096] For example, the virtual space manager (451) may include a runtime service (452). As an example, the runtime service (452) may be referred to as an OpenXR runtime module. The wearable device (200) may be used to provide at least one of a pose prediction function, a frame timing function, and / or a spatial input function to a user through the wearable device (200) based on the execution of the runtime service (452). As an example, the wearable device (200) may be used to perform rendering for a virtual space service to a user based on the execution of the runtime service (452). For example, an application (e.g., unity or an OpenXR native application) may be implemented based on the execution of the runtime service (452).

[0097] For example, the virtual space manager (451) may include a pass-through manager (453). Based on the execution of the pass-through manager (453), the wearable device (200) may display a screen representing a virtual space on the display (420), while another screen representing an actual space acquired through the camera (425) may be superimposed on at least a portion of the screen.

[0098] For example, the virtual space manager (451) may include an input manager (454). Based on the execution of the input manager (454), the wearable device (200) may identify data (e.g., sensor data) acquired by executing one or more programs included in the recognition service layer (470). The wearable device (200) may initiate execution of at least one of the functions of the wearable device (200) using the acquired data.

[0099] For example, the perception abstract layer (460) can be used for data exchange between the virtual space manager (451) and the perception service layer (470). From the perspective of being used for data exchange between the virtual space manager (451) and the perception service layer (470), the perception abstract layer (460) can be referred to as an interface. For example, the perception abstract layer (460) can be referenced as OpenPX. The perception abstract layer (460) can be used for a perception client and a perception service.

[0100] According to one embodiment, the recognition service layer (470) may include one or more programs for processing data acquired from a sensor (430) (or a camera (425)). The one or more programs may include at least one of a position tracker (471), a space recognizer (472), a gesture tracker (473), an eye tracker (474), and / or a face tracker (473). The type and / or number of the one or more programs included in the recognition service layer (470) are not limited to those illustrated in FIG. 4.

[0101] For example, the wearable device (200) can identify the pose of the wearable device (200) using the sensor (430) based on the execution of the position tracker (471). The wearable device (200) can identify the 6 degrees of freedom pose (6 dof pose) of the wearable device (200) using data acquired using the camera (425) and the IMU based on the execution of the position tracker (471). The position tracker (471) can be referred to as a head tracking (HeT) module.

[0102] For example, the wearable device (200) may be used to construct a three-dimensional virtual space around the wearable device (200) (or a user of the wearable device (200)) based on the execution of the space recognizer (472). The wearable device (200) may reconstruct the three-dimensional surroundings of the wearable device (200) using data acquired using the camera (425) based on the execution of the space recognizer (472). The wearable device (200) may identify at least one of a plane, a slope, and stairs based on the three-dimensionally reconstructed surroundings of the wearable device (200) based on the execution of the space recognizer (472). The space recognizer (472) may be referred to as a scene understanding (SU) module.

[0103] For example, the wearable device (200) may be used to identify (or recognize) a pose and / or gesture of a hand of a user of the wearable device (200) based on the execution of the gesture tracker (473). As an example, the wearable device (200) may identify a pose and / or gesture of a hand of a user using data acquired from a sensor (430) based on the execution of the gesture tracker (473). As an example, the wearable device (200) may identify a pose and / or gesture of a hand of a user based on data (or images) acquired using a camera based on the execution of the gesture tracker (473). The gesture tracker (473) may be referred to as a hand tracking (HaT) module and / or a gesture tracking module.

[0104] For example, the wearable device (200) may identify (or track) eye movements of a user of the wearable device (200) based on the execution of the gaze tracker (474). As an example, the wearable device (200) may identify eye movements of the user using data acquired from at least one sensor based on the execution of the gaze tracker (474). As an example, the wearable device (200) may identify eye movements of the user based on data acquired using a camera (e.g., the gaze tracking camera (260-1) of FIGS. 2A and 2B) and / or an infrared light emitting diode (IR LED) based on the execution of the gaze tracker (474). The gaze tracker (474) may be referred to as an eye tracking (ET) module and / or a gaze tracking module.

[0105] For example, the recognition service layer (470) of the wearable device (200) may further include a face tracker (475) for tracking the user's face. For example, the wearable device (200) may identify (or track) the movement of the user's face and / or the user's expression based on the execution of the face tracker (475). The wearable device (200) may estimate the user's expression based on the movement of the user's face based on the execution of the face tracker (475). As an example, the wearable device (200) may identify the movement of the user's face and / or the user's expression based on data (e.g., an image) acquired using a camera based on the execution of the face tracker (475).

[0106] For the embodiments described below, the wearable device (200) of FIG. 4 may be referenced. For example, the embodiments described below may be performed by the processor (410) of the wearable device (200) of FIG. 4.

[0107] FIG. 5 illustrates examples of virtual spaces that change depending on the display mode of the display, according to one embodiment.

[0108] Referring to FIG. 5, the wearable device (200) may correspond to the wearable device (200) illustrated in FIG. 4. For example, the wearable device (200) may include a camera (425) positioned toward the front of the user (510) when the device is worn by the user (510). The front of the user (510) may include the head of the user (510) and / or the direction in which the gaze of the user (510) is directed. According to one embodiment, the wearable device (200) may include a sensor (430) for identifying the motion of the head of the user (510) and / or the wearable device (200) when the device is worn by the user (510).

[0109] The processor (410) of the wearable device (200) can identify the angle of the wearable device (200) based on data from the sensor (430). In order to provide a user interface (UI) based on virtual reality (VR), augmented reality (AR), and / or mixed reality (MR) to a user (510) wearing the wearable device (200), the processor (410) can control the camera (425) and / or the sensor (430). The UI can be related to the wearable device (200), a metaverse service provided by a server connected to the wearable device (200), and / or a notification service.

[0110] According to one embodiment, the processor (410) may identify the display mode of the display (420) as one of a first display mode and a second display mode. For example, the processor (410) may display a first virtual space (501) (or a portion of the first virtual space (501)) through the display (420) based on the first display mode. The processor (410) may display a second virtual space (502) (or a portion of the second virtual space (502)) through the display (420) based on the second display mode.

[0111] For example, a first virtual space (501) according to a first display mode may be configured to be distinct from a space (i.e., an actual space) in which a user (510) is located. A second virtual space (502) according to a second display mode may be configured to include a space (i.e., an actual space) in which a user (510) is located and one or more visual objects displayed in an overlapping manner in the space.

[0112] According to one embodiment, the first display mode may include a mode for displaying (or providing) a virtual space (e.g., virtual space (501)) related to virtual reality (VR). For example, the wearable device (200) may include a display (420) disposed on a first side facing the eyes of the user (510) (e.g., the first side (310) of FIG. 3A). The wearable device (200) may include a camera (425) (or cameras (260-7, 260-8, 260-9, 260-10, 260-11, 260-12) of FIG. 3B) disposed on a second side opposite to the first side (e.g., the second side (320) of FIG. 3A). Using the camera (425), the processor (410) may acquire frame images including ambient light. The processor (410) can output frame images to the display (420) disposed on the first surface, so that the user (510) can recognize ambient light through the display (420). The display area of ​​the display (420) disposed on the first surface can be formed by one or more pixels included in the display (420). The processor (410) can synthesize a virtual object within the frame images output through the display (420) so that the user (510) can recognize the virtual object together with the actual object recognized by the ambient light.

[0113] According to one embodiment, the second display mode may include a mode for displaying (or providing) a virtual space (e.g., virtual space (502)) related to augmented reality (AR) and / or mixed reality (MR). While the user (510) is wearing the wearable device (200), the processor (410) may include at least one lens positioned adjacent to the eyes of the user (510). Ambient light passing through the lens may be combined (or mixed) with light emitted from the display (420) of the wearable device (200). The display area of ​​the display (420) may be formed within the lens through which the ambient light passes. Since the processor (410) combines the ambient light and the light emitted from the display (420), the user (510) can see an image that is a mixture of a real object recognized by the ambient light and a virtual object formed by the light emitted from the display (420).

[0114] According to one embodiment, the processor (410) may provide a user experience based on mixed reality (MR) while the display mode of the display (420) is set to one of the first display mode and the second display mode. The processor (410) of the wearable device (200) may recognize a space (e.g., a real space) in which the user (510) (or the wearable device (200)) is located, and generate a virtual space mapped to the space. The spatial recognition performed by the processor (410) may include simultaneous localization and mapping (SLAM) and / or spatial mapping (e.g., scene understanding).

[0115] According to one embodiment, the wearable device (200) may operate based on video seethrough (VST). For example, the processor (410) may display a virtual space (501) provided to the user (510) based on a video (or rendered image). For example, while the display mode of the display (420) is a first display mode, the processor (410) may generate a virtual space (501) and display the generated virtual space (501) through the display (420). For example, the virtual space (501) may be configured based on an external environment. The processor (410) may configure a virtual space (501) representing the external environment and display additional virtual visual objects within the virtual space (501). For example, while the display mode of the display (420) is a second display mode, the processor (410) may acquire an image of the external environment using the camera (425). The processor (410) can display a virtual space (502) including an image of an external environment and visual objects (e.g., visual objects (521) and visual objects (522)) superimposed on the image through the display (420). The visual objects superimposed on the image of the external environment can be set in various ways. For example, the visual objects can be composed of multimedia content (e.g., video).

[0116] According to one embodiment, the wearable device (200) may operate based on optical seethrough (OST). For example, the processor (410) may display a specified color (or a specified image) through the display (420) so as not to display the space where the wearable device (200) is located while the display mode of the display (420) is a first display mode. The processor (410) may display a virtual space (501) including a visual object superimposed on the specified color (or the specified image). For example, the processor (410) may display a virtual space (502) including an image in which a real object and a virtual object are mixed through the display (420) while the display mode of the display (420) is a second display mode.

[0117] According to one embodiment, when the display mode of the display (420) is changed from the first display mode to the second display mode, a change in brightness perceived by the user (510) may occur due to a change in the brightness of the screen displayed through the display (420) and / or the brightness of the external environment. According to one embodiment, when the display mode of the display (420) is changed from the second display mode to the first display mode, a change in brightness perceived by the user (510) may occur due to the brightness (or brightness value) set in the first display mode. Therefore, depending on the change in brightness perceived by the user (510), the user (510) may not be able to identify the content (e.g., a visual object) displayed through the display (420). For example, the user (510) may not be able to identify the content displayed through the display (420) during light adaptation or dark adaptation. In the following specification, the operation of the wearable device (200) (or the processor (410) of the wearable device (200)) to solve the problem of the user (510) not being able to identify content due to a change in the display mode of the display (420) will be described.

[0118] For example, the wearable device (200) may display a changed screen in the second display mode based on whether the difference in brightness between the screen displayed in the first display mode and the screen displayed in the second display mode is outside a specified range as the display mode of the display (420) changes from the first display mode to the second display mode.

[0119] Figure 6a illustrates a flowchart of the operation of a wearable device according to one embodiment. In the following embodiments, the operations may be performed sequentially, but are not necessarily sequential. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0120] Referring to FIG. 6A, in operation 610, the processor (410) may display a first screen regarding a first virtual space through the display (420) while the display mode of the display (420) is the first display mode. For example, the processor (410) may set the display mode of the display (420) to the first display mode. For example, the first display mode may be set based on VR.

[0121] According to one embodiment, since the virtual space is configured in three dimensions, the processor (410) may not be able to display the entire virtual space through the display (420). Therefore, the processor (410) may display the virtual space through the display (420) based on displaying a screen related to the virtual space through the display (420). For example, the processor (410) may display a first screen related to the first virtual space through the display (420). As an example, the processor (410) may display the first screen related to the first virtual space based on VR.

[0122] In operation 620, the processor (410) may identify information about the external environment. For example, the processor (410) may identify information about the external environment in which the wearable device (200) is located while at least one screen including a first screen relating to the first virtual space is displayed.

[0123] According to one embodiment, the processor (410) can display at least one screen including a first screen relating to a first virtual space.

[0124] For example, the processor (410) may change the first screen of the first virtual space to another screen based on the motion of the wearable device (200). The processor (410) may display at least one screen based on the change in the motion of the wearable device (200). The at least one screen may refer to screens that are continuously changed based on the change in the motion of the wearable device (200).

[0125] For example, the processor (410) may change the first screen regarding the first virtual space to another screen based on an application running on the wearable device (200) and / or a service provided by the wearable device (200). The processor (410) may display at least one screen based on at least one of the applications running on the wearable device (200) and / or the services provided by the wearable device (200). The at least one screen may mean screens that are continuously changed based on at least one of the applications running on the wearable device (200) and / or the services provided by the wearable device (200).

[0126] For example, the processor (410) may change the first screen of the first virtual space to another screen based on a signal identified in the wearable device (200). For example, the signal may be identified by a user input. For example, the signal may be received from another device. The processor (410) may display at least one screen based on the signal identified in the wearable device (200). The at least one screen may refer to screens that are continuously changed based on the signal identified in the wearable device (200).

[0127] According to one embodiment, the processor (410) may identify information about the external environment in which the wearable device (200) is located. For example, the information about the external environment may include information about the location of a light source within a space in which the wearable device (200) (or the user (510)) is located. For example, the information about the external environment may include information about the location of at least one light source within a space in which the wearable device (200) is located. The operation of the processor (410) for identifying information about the location of at least one light source within a space in which the wearable device (200) is located will be described in 7a.

[0128] For example, information about the external environment may include information about the movement of a light source (e.g., information about the direction of movement, information about the speed of movement). For example, information about the external environment may include information about the location of a light source having a brightness value greater than or equal to a specified value. For example, information about the external environment may include information about the location of the brightest light source among at least one light source within the space where the wearable device (200) is located. For example, information about the external environment may include information about the number of at least one light source within the space where the wearable device (200) is located. For example, information about the external environment may include illuminance information about the external environment.

[0129] At operation 630, the processor (410) may identify that the display mode of the display (420) is changed from the first display mode to the second display mode. For example, the processor (410) may identify that the display mode of the display (420) is changed from the first display mode to the second display mode after information about the external environment is identified.

[0130] According to one embodiment, the processor (410) may change the display mode of the display (420) from the first display mode to the second display mode based on satisfying a specified condition. For example, the processor (410) may identify whether the specified condition is satisfied based on a specified input, the magnitude of movement of the wearable device (200), the position of the wearable device (200), and / or the direction in which the wearable device (200) is facing. An example of a specific operation for identifying whether the processor (410) satisfies the specified condition will be specifically described in FIG. 8.

[0131] In operation 640, the processor (410) may identify a second screen related to a second virtual space according to a second display mode. For example, the processor (410) may identify a second screen related to a second virtual space according to a second display mode based on identifying that the display mode of the display (420) has changed from the first display mode to the second display mode. For example, the processor (410) may identify a second screen related to the second virtual space based on that the display mode of the display (420) is the second display mode. For example, the processor (410) may first identify the second screen, which is a screen to be displayed through the display (420).

[0132] In one embodiment, the second screen may be a screen that is recognized by the user. For example, the difference between the brightness (or illuminance perceived by the user) value of the second screen identified according to the second display mode and the brightness (or illuminance perceived by the user) value of the currently displayed screen may be greater than or equal to a specified value. When the second screen is displayed through the display (420), the user (510) may experience discomfort. For example, the processor (410) may identify the difference between the brightness value of an area related to the user's gaze within the currently displayed screen and the brightness value of an area related to the user's gaze within the second screen. If the difference in the brightness value is greater than or equal to a specified value, the user (510) may experience discomfort.

[0133] According to an embodiment, if the difference between the brightness value of the area related to the user's gaze within the currently displayed screen and the brightness value of the area related to the user's gaze in the second screen is less than a specified value, the user (510) may not experience any discomfort. Therefore, the processor (410) may display the second screen through the display (420) without performing operation 650. For example, the processor (410) may identify that the change in brightness of the area related to the user's gaze is within the user's contrast adaptable range. The processor (410) may display the second screen through the display (420) without performing operation 650. For example, the processor (410) may identify that the change in brightness of the area related to the user's gaze is outside the user's contrast adaptable range. The processor (410) may perform operation 650. For example, the user's contrast adaptable range may be identified through a specified model (e.g., an artificial intelligence model) indicated by a plurality of parameters. The processor (410) may learn a designated model based on at least one of the user's location, the time of using the wearable device (200), the application running on the wearable device (200), and the service provided through the wearable device (200). The processor (410) may identify the user's brightness adaptability range based on the designated model. For example, even when the user of the wearable device (200) is the same, the processor (410) may change the user's brightness adaptability range depending on the currently running application (or service provided). For example, the processor (410) may set the user's brightness adaptability range differently while providing a first service (e.g., a game service) and the user's brightness adaptability range differently while providing a second service (e.g., a service in which virtual objects are displayed in a real environment).

[0134] According to an embodiment, the processor (410) may track (or monitor) the user's gaze (or ROI). Based on changes in the user's gaze, the processor (410) may identify the user's gaze direction and angle of view. Based on the user's gaze direction and angle of view, the processor (410) may identify whether a light source is located in the user's field of view. The processor (410) may identify whether a change in brightness of an area of ​​the user's changed gaze falls within the user's brightness adaptability range.

[0135] In operation 650, the processor (410) may display a third screen changed from the second screen through the display (420) while the display mode of the display (420) is the second display mode. For example, the processor (410) may display a third screen changed from the second screen through the display (420) while the display mode of the display (420) is the second display mode, based on information about the external environment and a first area within the second screen identified according to the gaze of the user (510) of the wearable device (200).

[0136] According to one embodiment, the processor (410) can identify the gaze of a user (510) of the wearable device (200). The processor (410) can identify a first area within a second screen based on the gaze of the user (510). The processor (410) can identify a first area within a second screen to be displayed through the display (420) based on the gaze of the user (510).

[0137] According to one embodiment, the processor (410) may identify whether a first region within the second screen, identified according to the user's (510) gaze, corresponds to a second region within the second screen, with respect to a location where a light source is arranged. For example, the second region may be related to a location where one of the at least one light source is arranged. For example, the second region may include an area where the brightest light source among at least one light source (or multiple light sources) in the external environment is located. For example, the second region may include an area where the light source is finally identified according to the movement of the light source.

[0138] For example, the processor (410) may identify that the first region corresponds to the second region. Based on identifying that the first region corresponds to the second region, the processor (410) may display a third screen changed from the second screen through the display (420) while the display mode of the display (420) is the second display mode. A specific example of the processor (410) displaying the third screen changed from the second screen will be described in FIG. 9.

[0139] For example, the processor (410) can identify that the first region is distinct from the second region. Based on identifying that the first region is distinct from the second region, the processor (410) can display a second screen through the display (420).

[0140] According to one embodiment, the processor (410) may identify that the user's (510) gaze is positioned in a second area within the second screen corresponding to an external light source based on the display mode of the display (420) changing from a first display mode to a second display mode. The processor (410) may display a third screen changed from the second screen through the display (420) so that the user (510) does not perceive a change in brightness exceeding a specified value.

[0141] For example, the processor (410) may display a third screen changed from the second screen through the display (420) based on changing the position of a visual object displayed in the first area (or second area) of the second screen.

[0142] For example, the processor (410) can change data regarding the first area (or the second area) of the second screen. For example, the processor (410) can change data regarding the brightness of the first area. Based on changing the data regarding the brightness of the first area, the processor (410) can reduce the brightness of the first area. For example, the processor (410) can overlay a grayscale image on the first area. Based on overlaying the grayscale image on the first area, the processor (410) can reduce the brightness of the first area perceived by the user (510). For example, the processor (410) can reduce the brightness of the first area based on overlaying the grayscale image on the first area. Based on overlaying the grayscale image on the first area, the processor (410) can reduce the brightness of the first area to within a brightness adaptable range of the user.

[0143] According to one embodiment, while the display mode of the display (420) is the second display mode, the processor (410) may provide a virtual external space (720) in which a plurality of virtual visual objects are displayed together in an actual external environment. The processor (410) may overlay a grayscale image to change the brightness of each of the plurality of virtual visual objects. The processor (410) may overlay the grayscale image based on the distance between each of the plurality of virtual visual objects and a light source (or at least one light source). The grayscale image may be changed based on the distance between the virtual visual object and the light source. Therefore, the grayscale image overlaid on each of the plurality of virtual visual objects may not be the same. According to an embodiment, the processor (410) may overlay the grayscale image on only at least one visual object among the plurality of virtual visual objects.

[0144] According to an embodiment, the processor (410) may identify whether a visual object (or content) to be displayed is affected by a light source. For example, if the visual object to be displayed is not affected by a light source, the processor (410) may display the visual object by changing only the brightness and / or color. For example, in a wearable device (200) that operates based on VST, if the display mode of the display (420) is the first display mode, the visual object (or content) to be displayed may not be affected by a light source. The processor (410) may display the third screen by changing only the brightness and / or color of the visual object included in the second screen. According to an embodiment, the processor (410) may change not only the brightness and / or color of the visual object, but also the size of the visual object and / or the time for which the visual object is displayed.

[0145] In operations 610 to 650, examples are described in which the first display mode is set based on VR and the second display mode is set based on AR, but this is not limited thereto. Depending on the embodiment, the first display mode may be set based on AR and the second display mode may be set based on VR.

[0146] Figure 6b illustrates a flowchart of the operation of a wearable device according to one embodiment. In the following embodiments, the operations may be performed sequentially, but are not necessarily sequential. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0147] Referring to FIG. 6B, operations 610 to 650 of FIG. 6A may be related to operations of changing the second screen regarding the second virtual space to a third screen and displaying the third screen when the display mode of the display (420) is changed, but operations 661 to 669 of FIG. 6B may be related to operations of changing the fourth screen regarding the first virtual space to a fifth screen and displaying the fifth screen when the display mode of the display (420) is not changed.

[0148] In operation 661, the processor (410) may display a first screen regarding the first virtual space through the display (420) while the display mode of the display (420) is the first display mode. For example, operation 661 may correspond to operation 610 of FIG. 6A.

[0149] In operation 663, the processor (410) may identify information about the external environment. For example, operation 663 may correspond to operation 620 of FIG. 6A.

[0150] In operation 665, the processor (410) may identify whether a specified condition is satisfied. For example, the specified condition may be satisfied when a sudden change occurs in the screen displayed through the display (420).

[0151] For example, the specified condition may be set as whether a value regarding the movement of the wearable device (200) exceeds a specified value. The processor (410) may identify a value regarding the movement of the wearable device (200). For example, within a specified time of the wearable device (200), the greater the movement of the wearable device (200), the greater the value regarding the movement of the wearable device (200) may be identified. The processor (410) may identify whether the specified condition is satisfied based on identifying whether the value regarding the movement of the wearable device (200) exceeds a specified value.

[0152] For example, the specified condition may be set based on whether the location of the wearable device (200) is outside a specified range. For example, the wearable device (200) may provide a service while positioned within the specified range. The processor (410) may determine whether the specified condition is satisfied based on determining whether the location of the wearable device (200) is outside the specified range.

[0153] For example, the specified condition may be set based on whether the change in the external environment's illuminance exceeds a threshold range. For example, the processor (410) may determine whether the specified condition is satisfied based on whether the change in the external environment's illuminance exceeds a threshold range.

[0154] The above-described specified conditions may be set to be identical or similar to the conditions for changing the display mode of the display (420) from the first display mode to the second display mode.

[0155] In operation 667, the processor (410) may identify a fourth screen related to the first virtual space according to the first display mode. For example, the processor (410) may identify a fourth screen to be displayed through the display (420) while the display mode of the display (420) is maintained in the first display mode. For example, the processor (410) may first identify the fourth screen, which is a screen to be displayed through the display (420). For example, the difference between the brightness value of the fourth screen and the brightness value of the currently displayed screen may be greater than or equal to a specified value. If the fourth screen is displayed through the display (420), inconvenience may occur to the user (510). For example, the operation of the processor (410) identifying a fourth screen related to the first virtual space may correspond to the operation of the processor (410) identifying a second screen related to the second virtual space in operation 640 of FIG. 6A.

[0156] In operation 669, the processor (410) may display a fifth screen changed from the fourth screen. For example, the processor (410) may display a fifth screen changed from the fourth screen through the display (420) while the display mode of the display (420) is the first display mode based on information about the external environment and a first area within the fourth screen identified according to the gaze of the user (510) of the wearable device (200). For example, the operation of the processor (410) changing the fourth screen to the fifth screen may correspond to the operation of the processor (410) changing the second screen to the third screen in operation 650 of FIG. 6A.

[0157] Referring to operations 661 to 669, the processor (410) may identify that a value related to the movement of the wearable device (200) exceeds a specified value while the display mode of the display (420) is the first display mode. Based on identifying that the value related to the movement of the wearable device (200) exceeds the specified value, the processor (410) may change the fourth screen to be displayed through the display (420) to a fifth screen and display it.

[0158] According to an embodiment, the processor (410) may not perform an operation for changing the screen if the operation for changing the screen is repeatedly performed even when a specified condition is satisfied. For example, the processor (410) may not perform an operation for changing the screen based on identifying that the number of times the operation for changing the screen is performed within a reference time is greater than a specified number of times.

[0159] For example, even if the processor (410) identifies that the value related to the movement of the wearable device (200) exceeds a specified value, the processor (410) may not change the fourth screen to be displayed through the display (420) to a fifth screen. For example, the processor (410) may not change the fourth screen to be displayed through the display (420) to a fifth screen based on the fact that the value related to the movement of the wearable device (200) remains in a state exceeding a specified value. After the value related to the movement remains in a state exceeding a specified value, the processor (410) may change the fourth screen to be displayed through the display (420) to a fifth screen and display it based on the fact that the value related to the movement changes to be less than or equal to the specified value.

[0160] According to one embodiment, the processor (410) can identify whether the amount of change in illuminance is outside the brightness adaptation range. Based on the amount of change in illuminance identifying the brightness adaptation range, the processor (410) can change the brightness of all virtual visual objects displayed within the fourth screen. According to one embodiment, the processor (410) can identify whether there is a need to change the brightness within a certain area within the fourth screen. Based on the location of the light source, the processor (410) can change the fourth screen into a fifth screen by changing the brightness of only a certain area within the fourth screen. According to one embodiment, the operation of the processor (410) changing the brightness of only a certain area within the fourth screen can be performed differently depending on the number of light sources, the brightness of the light sources, and / or the user's line of sight.

[0161] In operations 661 to 669, an example is described in which the first display mode is set based on VR and the second display mode is set based on AR, but this is not limited thereto. Depending on the embodiment, the first display mode may be set based on AR and the second display mode may be set based on VR.

[0162] Referring to the above-described operations 661 to 669, the processor (410) may change the screen to be displayed based on whether a specified condition is satisfied, even if the display mode of the display (420) is not changed.

[0163] For convenience of explanation, the following specification may describe embodiments for changing the screen to be displayed according to the operation of FIG. 6a. However, this is for convenience of explanation, and the embodiments described below may also be applied to embodiments for changing the screen according to the operation of FIG. 6b.

[0164] FIG. 7A illustrates a flowchart of the operations of a wearable device according to one embodiment. In the following embodiments, the operations may be performed sequentially, but are not necessarily sequential. For example, the order of the operations may be changed, and at least two operations may be performed in parallel. Operations 710 and 720 illustrated in FIG. 7A may correspond to operation 620 of FIG. 6A.

[0165] Referring to FIG. 7A, in operation 710, the processor (410) may identify information regarding the brightness of each of at least one screen. For example, the processor (410) may display a first screen regarding a first virtual space through the display (420) while the display mode of the display (420) is a first display mode. The processor (410) may display at least one screen including the first screen regarding the first virtual space. For example, the at least one screen may refer to screens that continuously change based on changes in the motion of the wearable device (200).

[0166] According to one embodiment, while the display mode of the display (420) is the second display mode, the external environment may be displayed through at least one screen. The at least one screen may be related to the external environment. If the at least one screen is related to the external environment, the processor (410) may acquire one or more images of the external environment through the camera (425) to display the at least one screen. Based on the images of the external environment, the processor (410) may acquire information about the exposure value and / or brightness. For example, based on the images of the external environment, the processor (410) may acquire information about the brightness according to the location of the external environment. The processor (410) may identify the brightness that changes according to the location of the external environment.

[0167] According to one embodiment, while the display mode of the display (420) is the first display mode, a virtual space distinct from the external environment may be displayed through at least one screen. The at least one screen may not be related to the external environment. If the at least one screen is not related to the external environment, the processor (410) may obtain information about the external environment while the at least one screen is displayed. To obtain information about the external environment, the processor (410) may obtain one or more images of the external environment. Based on the one or more images of the external environment, the processor (410) may obtain information about the exposure value and / or brightness.

[0168] According to one embodiment, the processor (410) may acquire one or more images of the external environment based on the gaze of the user (510). For example, the processor (410) may acquire one or more images of the external environment based on a change in the gaze of the user (510).

[0169] In operation 720, the processor (410) may identify information regarding the location of a light source within the space where the wearable device (200) is located. For example, the processor (410) may identify information regarding the external environment, including information regarding the location of a light source within the space where the wearable device (200) is located.

[0170] According to one embodiment, the processor (410) may configure the external environment as a three-dimensional virtual space. For example, the processor (410) may identify the location where a light source is placed within the external environment configured as a three-dimensional virtual space. For example, one or more light sources may exist within the external environment. The processor (410) may identify the location where one or more light sources are placed within the external environment configured as a three-dimensional virtual space.

[0171] For example, the processor (410) can identify information regarding brightness of one or more images of the external environment acquired through the camera (425). Based on the information regarding brightness of one or more images of the external environment, the processor (410) can identify information regarding a location where a light source is placed within a space where the wearable device (200) is located. For example, the processor (410) can identify that a light source is placed based on identifying that a value regarding brightness within one or more images is greater than or equal to a reference value. The processor (410) can identify not only the location where the light source is placed but also the brightness within the space where the wearable device (200) is located.

[0172] According to one embodiment, the processor (410) may identify information about the brightness distribution of the external environment and / or information about the maximum brightness based on information about the external environment. For example, one or more images about the external environment may be acquired based on an exposure value set in the camera (425). The processor (410) may identify information about the actual brightness of the external environment (e.g., information about the brightness distribution of the external environment and information about the maximum brightness) through one or more images based on the exposure value. For example, the brightness of the external environment may be identified as brightness within the image based on the exposure value set in the camera (425). The processor (410) may acquire a preview image. The processor (410) may identify information about the actual brightness of the external environment based on identifying the brightness of the preview image and the exposure value set for acquiring the preview image. The processor (410) may acquire a plurality of images (or videos). The processor (410) can identify information about the actual brightness of the external environment based on identifying brightness information (e.g., brightness values) according to the acquired plurality of images and the exposure value set for acquiring the plurality of images.

[0173] Accordingly, the processor (410) can identify (or estimate) information about the actual brightness of the external environment based on information about the brightness of one or more images and the exposure value set in the camera (425).

[0174] According to one embodiment, the processor (410) may obtain a distribution image of brightness values ​​(e.g., absolute values ​​of brightness) of the external environment based on information about the external environment. The distribution image of brightness values ​​of the external environment may refer to an image representing absolute values ​​of brightness of the external environment in a three-dimensional space. According to an embodiment, the distribution image of brightness values ​​of the external environment may be configured to represent the absolute value of brightness of one point (or area) within the image (or external environment).

[0175] According to one embodiment, the processor (410) may obtain information about a brightness value recognized by the user (510) based on a distribution image of brightness values ​​in an external environment. When a screen is output through the display (420) based on the distribution image of brightness values ​​in an external environment, the processor (410) may obtain information about a brightness value recognized by the user (510). For example, the processor (410) may change a brightness value recognized by the user (510) based on the size of the pupil of the user (510).

[0176] For example, the processor (410) can identify information about the size of the pupil through the camera (425) (e.g., a camera facing the eye of the user (510)). The processor (410) can obtain information about the brightness perceived by the user (510) based on the information about the size of the pupil and the information about the brightness of the screen to be output through the display (420). For example, in a wearable device (200) based on OST, when the display mode of the display (420) operates in the second display mode (e.g., a display mode based on AR), external light according to the external environment and light emitted through the display (420) of the wearable device (200) can be identified through the eyes of the user (510). Depending on the external light and the light emitted through the display (420), the size of the pupil of the user (510) can change. The processor (410) can identify that a dark screen is being provided to the user (510) based on identifying that the size of the user's (510) pupil is greater than or equal to a specified size. The processor (410) can identify that a bright screen is being provided to the user (510) based on identifying that the size of the user's (510) pupil is less than or equal to a specified size. According to an embodiment, the processor (410) can change a brightness value perceived by the user (510) based on the size of the user's (510) pupil. For example, the processor (410) can change the brightness value perceived by the user (510) by changing the amount of light provided through the display (420) based on the size of the user's (510) pupil.

[0177] According to an embodiment, the processor (410) may train a designated model (e.g., an artificial intelligence model) indicated by a plurality of parameters based on information about the pupil size of the user (510). The processor (410) may set the pupil size for changing the brightness value recognized by the user (510) based on the designated model. For example, the processor (410) may identify the preference (e.g., priority) of the user (510) based on the type of service provided through the wearable device (200) and / or the type of application running on the wearable device (200). The processor (410) may set the preference of the user (510) as an input value of the designated model. The processor (410) may identify the pupil size for changing the brightness value recognized by the user (510) based on the output value of the designated model.

[0178] According to an embodiment, the processor (410) may set at least one of the user's (510) preference and / or the user's brightness adaptability range as an input value of a specified model. Based on the output value of the specified model, the processor (410) may identify the size of the pupil for changing the brightness value perceived by the user (510).

[0179] According to one embodiment, the processor (410) can identify information about a screen to be displayed through the display (420). For example, the processor (410) can identify at least one visual object to be displayed on the screen based on a currently running application. The processor (410) can identify a screen on which at least one visual object is displayed. The processor (410) can identify a screen on which at least one visual object is displayed based on an operating mode of the wearable device (200) (e.g., VST and OST) and a display mode of the display (420) (e.g., a first display mode and a second display mode). The processor (410) can identify a screen to be recognized by the user (510) based on the information about the screen to be displayed through the display (420).

[0180] For example, in a wearable device (200) that operates based on VST, when the display mode of the display (420) is set to the first display mode (e.g., the display mode of the VR-based display (420), the processor (410) can display a screen regarding the generated virtual space regardless of the external environment. The processor (410) can identify that the screen displayed through the display (420) is a screen to be recognized by the user (510). For example, the processor (410) can track (or monitor) the user's gaze (or ROI). The processor (410) can identify that the screen displayed through the display (420) is a screen to be recognized by the user (510) based on the user's gaze direction and viewing angle.

[0181] For example, in a wearable device (200) that operates based on VST, when the display mode of the display (420) is set to a second display mode (e.g., a display mode of the AR-based display (420)), the processor (410) may identify a screen on which at least one visual object is displayed (e.g., overlaid) on a screen regarding an external environment acquired using an externally exposed camera (425) of the wearable device (200). According to an embodiment, the screen regarding the external environment may be acquired based on not only the externally exposed camera (425), but also illumination information acquired through an illumination sensor included in the sensor (430) and / or information regarding the external environment received from an external electronic device.

[0182] The processor (410) can identify a screen on which at least one visual object is displayed (e.g., overlaid) on a screen related to an external environment as a screen to be displayed through the display (420). The processor (410) can identify a screen on which at least one visual object is displayed on a screen related to an external environment as a screen to be recognized by a user (510).

[0183] For example, in a wearable device (200) that operates based on OST, when the display mode of the display (420) is set to the first display mode, the processor (410) may identify a screen on which a virtual background for obscuring an external environment and at least one visual object placed on the virtual background are displayed as a screen to be displayed through the display (420). The processor (410) may identify the screen on which the virtual background and at least one visual object are displayed as a screen to be recognized by the user.

[0184] For example, in a wearable device (200) that operates based on OST, when the display mode of the display (420) is set to the second display mode, the external environment can be shown to the user (510) as it is. The processor (410) can identify a screen for displaying a visual object that is overlapped with the external environment as a screen to be displayed through the display (420). The processor (410) can identify a screen on which at least one visual object is displayed in the external environment as a screen to be recognized by the user (510). According to the above-described embodiment, the processor (410) can identify a screen to be recognized by the user (510) and store the screen to be recognized by the user (510) in the memory (415).

[0185] According to one embodiment, the processor (410) can identify in advance a screen to be displayed when the display mode of the display (420) is changed from a first display mode to a second display mode. In response to identifying that the display mode of the display (420) is changed from the first display mode to the second display mode, the processor (410) can display the pre-identified screen.

[0186] According to one embodiment, the processor (410) may repeatedly perform the above-described operations (e.g., operations 710 and 720). For example, the processor (410) may identify in advance a screen to be displayed when the display mode of the display (420) is changed from a first display mode to a second display mode based on a specified cycle. For example, the processor (410) may identify in advance a screen to be displayed when the display mode of the display (420) is changed from a first display mode to a second display mode based on a value related to the movement of the wearable device (200) exceeding a specified value. For example, the processor (410) may identify in advance a screen to be displayed when the display mode of the display (420) is changed from a first display mode to a second display mode based on a change in the illumination of the external environment exceeding a threshold range.

[0187] According to one embodiment, the processor (410) may identify a screen to be displayed through the display (420) as the second screen described in FIG. 6A. The processor (410) may compare a screen to be recognized by the user (510) and a screen (or second screen) to be displayed when the display mode of the display (420) is changed from the first display mode to the second display mode. Based on comparing the screen to be recognized by the user (510) and the screen (or second screen) to be displayed when the display mode of the display (420) is changed from the first display mode to the second display mode, the processor (410) may identify that a difference in brightness (or a difference in illuminance recognized by the user (510)) occurs in an area corresponding to the line of sight of the user (510). The processor (410) may perform an operation to change the second screen to a third screen.

[0188] FIG. 7b illustrates an example of an operation for identifying a location where a light source is placed within a space where a wearable device is located, according to one embodiment.

[0189] Referring to FIG. 7B, the processor (410) may identify one or more images of the external environment. For example, the processor (410) may acquire one or more images of the external environment via a camera (425) (e.g., a camera facing the external environment). Based on identifying an area within one or more images where a brightness value is greater than a reference value, the processor (410) may identify that a light source is positioned in the identified area.

[0190] For example, the processor (410) can perform three-dimensional modeling of the space in which the wearable device (200) is located. The processor (410) can identify a virtual external space (750) based on the three-dimensional modeling. The processor (410) can identify a location where a light source is placed within the virtual external space (750).

[0191] The processor (410) can identify that four light sources are positioned within a virtual external space (750) based on one or more images of the external environment. The processor (410) can identify the locations of the four light sources within the virtual external space (750).

[0192] For example, the processor (410) can identify light generated by a light bulb (or electric bulb). The processor (410) can identify light sources (751), light sources (752), and light sources (753) within a virtual external space (750). The processor (410) can identify light sources (751), light sources (752), and light sources (753) that emit light from a single point.

[0193] For example, the processor (410) can identify light generated by sunlight entering a space where a wearable device (200) is located through a window. The processor (410) can identify a light source (754) within a virtual external space (750). The processor (410) can identify a light source (754) emitting light from a designated area.

[0194] According to one embodiment, the processor (410) may identify information about the external environment in which the wearable device (200) is located while at least one screen including the first screen is displayed. For example, the information about the external environment may include a virtual external space (750) including at least one light source and an image of a distribution of brightness values ​​of the external environment. For example, the information about the external environment may include at least one of information about the number of at least one light source, information about the brightest light source among the at least one light source, information about movement of the at least one light source, and information about illuminance according to the at least one light source.

[0195] For example, the processor (410) may configure a virtual external space (750) including a light source (751), a light source (752), a light source (753), and a light source (754). The processor (410) may identify a distribution image of brightness values ​​of an external environment based on the virtual external space (750). For example, the distribution image of brightness values ​​of the external environment may represent the absolute value of brightness at each point within the virtual external space (750). For example, the distribution image of brightness values ​​of the external environment may be identified based on a graph representing brightness values ​​according to a location of the virtual external space (750).

[0196] According to one embodiment, the processor (410) can identify a screen to be displayed based on a running application. The processor (410) can compensate for the screen to be displayed based on information about the external environment. For example, the processor (410) can compensate for the screen to be displayed so that the screen is displayed based on illumination similar to the external environment. For example, the processor (410) can store the screen to be displayed in the memory (415). In response to identifying that the display mode of the display (420) has changed from a first display mode to a second display mode, the processor (410) can display the screen stored in the memory (415).

[0197] In one embodiment, the processor (410) can identify that a light source (e.g., light source (751)) is moving within an external environment. The processor (410) can identify a movement path of the light source and / or a movement speed of the light source. The processor (410) can correct the screen based on the movement path of the light source and / or the movement speed of the light source.

[0198] According to one embodiment, the processor (410) can identify the brightest light source among a plurality of light sources within an external environment. For example, the processor (410) can identify whether the brightest light source within the external environment changes. The processor (410) can calibrate the screen based on the brightest light source among the plurality of light sources within the external environment.

[0199] In one embodiment, the processor (410) can identify illuminance within an external environment. For example, the processor (410) can compensate for a screen based on identifying that illuminance within the external environment has changed.

[0200] According to one embodiment, when the screen to be displayed is corrected, the user's (510) profile may be used. For example, the user's (510) profile may include at least one of information about the user's (510) ability to adapt to light and dark, information about the difference in brightness between pixels that can be distinguished by the user's (510), information about the difference in color between pixels that can be distinguished by the user's (510), and information about color weakness. The processor (410) may correct the screen to be displayed based on the user's (510) profile. For example, the user's (510) profile may be input from the user's (510) or identified based on an action performed by the user's (510). For example, the user's (510) profile may also be identified based on physical information (e.g., age and gender) of the user's (510).

[0201] For example, the processor (410) may calibrate the screen to be displayed based on a designated model (e.g., an artificial intelligence model) indicated by a plurality of parameters. For example, the designated model may be configured according to the user. The designated model may be updated based on the user's (510) usage history of the wearable device (200). In some embodiments, the user's (510) profile may be set as an input value of the designated model.

[0202] According to the above-described embodiment, the processor (410) can identify the second screen described in FIG. 6A based on correcting the screen to be displayed.

[0203] Figure 8 illustrates a flowchart of the operation of a wearable device according to one embodiment. In the following embodiments, the operations may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel. Operations 810 to 830 may be related to operation 630 of Figure 6A.

[0204] In operation 810, the processor (410) may determine whether a specified condition is satisfied. For example, the processor (410) may identify information about the external environment and then determine whether the specified condition is satisfied. For example, the specified condition may be used to determine whether to change the display mode of the display (420).

[0205] For example, the specified condition may be set based on whether a specified input is received. The specified input may include an input for changing the display mode of the display (420). The processor (410) may determine whether the specified condition is satisfied based on determining whether the specified input is received.

[0206] For example, the specified condition may be set as whether a value regarding the movement of the wearable device (200) exceeds a specified value. The processor (410) may identify a value regarding the movement of the wearable device (200). For example, within a specified time of the wearable device (200), the greater the movement of the wearable device (200), the greater the value regarding the movement of the wearable device (200) may be identified. The processor (410) may identify whether the specified condition is satisfied based on identifying whether the value regarding the movement of the wearable device (200) exceeds a specified value.

[0207] For example, the specified condition may be set based on whether the location of the wearable device (200) is outside a specified range. For example, the wearable device (200) may provide a service while positioned within the specified range. The processor (410) may determine whether the specified condition is satisfied based on determining whether the location of the wearable device (200) is outside the specified range.

[0208] For example, the specified condition may be set based on whether the change in the external environment's illuminance exceeds a threshold range. For example, the processor (410) may determine whether the specified condition is satisfied based on whether the change in the external environment's illuminance exceeds a threshold range.

[0209] For example, the specified condition may be set based on whether the number of light sources in the external environment changes. For example, the processor (410) may determine whether the specified condition is satisfied based on whether the number of light sources in the external environment exceeds the specified number.

[0210] For example, the specified condition may be set to whether a change in illuminance occurs due to movement of a light source. For example, the processor (410) may identify whether the specified condition is satisfied based on identifying whether a change in illuminance occurs due to movement of the light source.

[0211] Depending on the embodiment, multiple conditions among the examples of the conditions described above may be combined to form a designated condition. When multiple conditions are combined to form a designated condition, the respective judgment criteria may be set lower.

[0212] In operation 820, if a specified condition is satisfied, the processor (410) may change the display mode of the display (420) from the first display mode to the second display mode. For example, the processor (410) may change the display mode of the display (420) from the first display mode to the second display mode based on identifying that the specified condition is satisfied.

[0213] In operation 830, if the specified condition is not satisfied, the processor (410) may maintain the display mode of the display (420) in the first display mode. The processor (410) may maintain the display mode of the display (420) in the first display mode based on identifying that the specified condition is not satisfied.

[0214] Although operations 810 to 830 illustrate examples in which the display mode of the display (420) changes from the first display mode to the second display mode when a condition specified in the first display mode is satisfied, the present invention is not limited thereto. For example, the display mode of the display (420) may change from the second display mode to the first display mode when a condition specified in the second display mode is satisfied.

[0215] Figure 9 illustrates a flowchart of the operation of a wearable device according to one embodiment. In the following embodiments, the operations may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel. Operations 910 to 950 may be related to operation 650 of Figure 6A.

[0216] Referring to FIG. 9, before operation 910 is performed, the processor (410) can identify that a first area within a second screen identified according to the user's (510) gaze corresponds to a second area within a second screen with respect to a location where a light source is placed. Based on identifying that the first area corresponds to the second area, the processor (410) can perform operation 910.

[0217] In operation 910, the processor (410) may identify whether a value related to a change in illuminance falls outside a first range. For example, the processor (410) may identify a value related to a change in illuminance between illuminance (e.g., average illuminance, maximum illuminance) related to the first screen and illuminance related to the second screen. The processor (410) may identify whether the value related to the change in illuminance falls outside the first range. The illuminance related to the first screen described above may refer to an illuminance value recognized by the user (510) based on the display of the first screen. The illuminance related to the second screen described above may refer to an illuminance value recognized by the user (510) based on the display of the second screen.

[0218] For example, the processor (410) can identify whether a value related to a change in illuminance in a first region corresponding to a second region is outside a first range. The processor (410) can identify whether a value related to a change in illuminance in a first region identified according to the line of sight of the user (510) is outside a first range. The processor (410) can identify whether a significant change in illuminance occurs at the position of the line of sight of the user (510). If a significant change in illuminance occurs at the position of the line of sight of the user (510), it may cause discomfort to the user (510). Therefore, the processor (410) can identify whether a value related to a change in illuminance is outside a first range.

[0219] For example, the first range may be set based on a designated model (e.g., an artificial intelligence model) indicated by a plurality of parameters. The processor (410) may change the first range based on learning data regarding changes in illumination as the display mode of the display (420) changes. For example, the designated model may be configured according to the user. The designated model may be updated based on the user's usage history of the wearable device (200).

[0220] In operation 920, if the value related to the change in illuminance does not fall outside the first range, the processor (410) may display a second screen via the display (420). For example, the second screen may be a screen that has not changed based on information about the external environment. Based on determining that the value related to the change in illuminance does not fall outside the first range, the processor (410) may display the second screen via the display (420).

[0221] In operation 930, if the value related to the illuminance change is outside the first range, the processor (410) may identify whether the value related to the brightness of the light source is outside the second range. For example, the processor (410) may identify whether the value related to the brightness of the light source is outside the second range based on identifying that the value related to the illuminance change is outside the first range.

[0222] For example, the processor (410) may be associated with a location where a light source is placed in a second area within a second screen. The processor (410) may identify whether a value related to the brightness of a light source located in the second area is outside a second range.

[0223] According to one embodiment, the processor (410) may display a visual object within a first region corresponding to a second region. The processor (410) may display a visual object within the first region where the user's (510) gaze is located. For example, if a value related to a change in illuminance within the first region is outside a first range, the visual object may not be recognized by the user (510) due to the illuminance change. In order to increase the visibility of the visual object, the processor (410) may identify whether a value related to the brightness of a light source located in the second region is outside the second range. The processor (410) may increase the visibility of the visual object through operations 940 and 950.

[0224] For example, the processor (410) may determine that the brightness of a light source falls outside the range that the display (420) can express. For example, if the brightness value of the light source is greater than the brightness value that the engine of the display (420) can output, the visibility of content in the area corresponding to the light source may decrease. Accordingly, the processor (410) may identify whether the value related to the brightness of the light source falls outside the second range.

[0225] In operation 940, if the value related to the brightness of the light source falls outside the second range, the processor (410) may change the position of the visual object displayed in the first area within the second screen. For example, the processor (410) may change the position of the visual object displayed in the first area within the second screen based on identifying that the value related to the brightness of the light source falls outside the second range.

[0226] Even if the processor (410) reduces the brightness of the second region corresponding to the light source when the value of the brightness of the light source in the second region (or the first region) falls outside the second range, the visibility of the visual object may not be good. Accordingly, the processor (410) may change the location of the visual object from the first region corresponding to the second region to another region. For example, the processor (410) may display an element (e.g., an arrow) indicating that the location of the visual object has been changed together with the visual object.

[0227] According to an embodiment, the processor (410) may display a visual object in a different region, distinct from the first region corresponding to the second region, in order to change the position of the user's (510) gaze (or region of interest (ROI)). For example, the processor (410) may track (or monitor) the user's gaze (or ROI). Based on the user's gaze direction and viewing angle, the processor (410) may identify whether the user (510) is looking at the first region within the second screen. If the user (510) looks at the first region within the second screen, the user may look at a light source. This may cause discomfort to the user (510). Therefore, the processor (410) may display a visual object in a different region, distinct from the first region corresponding to the second region, in order to change the position of the user's gaze. The processor (410) can induce the user's (510) gaze to move by displaying a visual object in a different area that is distinct from the first area.

[0228] In operation 950, if the value regarding the brightness of the light source does not fall outside the second range, the processor (410) may change the data for the first region. For example, the processor (410) may change the data for the first region based on identifying that the value regarding the brightness of the light source does not fall outside the second range.

[0229] The processor (410) may reduce the brightness of the first region (or the illuminance perceived by the user (510) in the first region) when the value of the brightness of the light source in the first region (or the second region) is outside the second range. The processor (410) may apply a filter for adjusting the light source in the first region.

[0230] For example, the processor (410) may change data for the first area based on a filter for adjusting a light source. By changing the data for the first area, the processor (410) may reduce the brightness of the first area of ​​the second screen. Based on reducing the brightness of the first area of ​​the second screen, the processor (410) may reduce the illuminance perceived by the user (510) in the first area of ​​the first screen.

[0231] For example, the processor (410) may display a filter for adjusting a light source on a second screen of a second virtual space displayed through the display (420). The filter for adjusting a light source may be used to block at least a portion of light emitted from a light source. As an example, the filter for adjusting a light source may include a grayscale image. The processor (410) may display the grayscale image in the first area to reduce the illuminance perceived by the user (510) in the first area. As an example, the processor (410) may reduce the brightness of the first area based on overlaying the grayscale image on the first area. The processor (410) may reduce the brightness of the first area to within a range of brightness adaptability of the user based on overlaying the grayscale image on the first area.

[0232] According to an embodiment, the processor (410) may apply a filter for adjusting the light source to a first region corresponding to a second region while maintaining the position of the user's (510) gaze (or region of interest (ROI)). When the user (510) looks at the first region, he / she may look at the light source. This may cause discomfort to the user (510). Therefore, the processor (410) may apply a filter for adjusting the light source to the first region to prevent a sudden change in the illuminance in the first region.

[0233] FIG. 10 illustrates an example of operation of a wearable device according to one embodiment.

[0234] Referring to FIG. 10, the processor (410) can identify that the display mode of the display (420) has changed from the first display mode to the second display mode. Based on identifying that the display mode of the display (420) has changed from the first display mode to the second display mode, the processor (410) can identify a second screen (1010) related to the second virtual space according to the second display mode. For example, the processor (410) can identify the second screen (1010) based on operation 640 of FIG. 6A.

[0235] According to one embodiment, the processor (410) may identify a first area within the second screen (1010) based on the gaze of the user (510). The processor (410) may identify a second area within the second screen (1010) with respect to a location where a light source (1015) is arranged. For example, the processor (410) may identify that the first area corresponds to the second area. The processor (410) may identify that the first area corresponds to the area (1011) and that the second area corresponds to the area (1011). For example, the light source (1015) may be displayed in the area (1011) within the second screen (1010). The gaze of the user (510) may be located in the area (1011) within the second screen (1010).

[0236] According to one embodiment, within the second screen (1010), a visual object (1012) may be displayed on an area (1011). The processor (410) may display the visual object (1012) on an area (1011) where the gaze of the user (510) is located. For example, the visual object (1012) may indicate that the display mode of the display (420) has been changed. For example, when the visual object (1012) is displayed on an area (1011) where a light source (1015) is arranged, visibility may decrease. The processor (410) may increase the visibility of the visual object (1012) by changing data for the first area.

[0237] According to an embodiment, the processor (410) may change the data for the first region based on identifying that the value for the brightness of the light source (1015) does not fall outside the second range.

[0238] According to one embodiment, the processor (410) may change the second screen (1010) to the third screen (1020) based on identifying that a visual object (1012) is displayed on an area (1011) where a light source (1015) is placed.

[0239] For example, the processor (410) may change the second screen (1010) to the third screen (1020) based on changing the data for the area (1011). As an example, the processor (410) may apply a filter for adjusting a light source to the area (1011). The processor (410) may change the data for the area (1011) based on applying the filter for adjusting a light source to the area (1011). The processor (410) may change the brightness (or illuminance perceived by the user) of the area (1011) based on changing the data for the area (1011).

[0240] For example, the processor (410) may display a filter (e.g., a grayscale image) for adjusting a light source in the area (1011). For example, the filter for adjusting a light source may be used to block at least a portion of light emitted from the light source. Based on displaying the filter for adjusting a light source in the area (1011), the processor (410) may change the second screen (1010) to the third screen (1020).

[0241] According to one embodiment, the processor (410) may improve the visibility of a visual object (1012) within a third screen (1020) as a filter for adjusting a light source is applied (or displayed).

[0242] Although the above-described embodiment has described an example in which a filter for adjusting a light source is applied (or displayed) to an area (1011) to improve the visibility of a visual object (1012), the present invention is not limited thereto. For example, even when a visual object (1012) is not displayed, the processor (410) may apply a light adjustment filter to an area (1011) to reduce a change in illumination that exceeds a threshold range recognized by a user (510).

[0243] FIG. 11 illustrates an example of operation of a wearable device according to one embodiment.

[0244] Referring to FIG. 11, the processor (410) can identify that the display mode of the display (420) has changed from the first display mode to the second display mode. Based on identifying that the display mode of the display (420) has changed from the first display mode to the second display mode, the processor (410) can identify a second screen (1110) related to the second virtual space according to the second display mode. For example, the processor (410) can identify the second screen (1110) based on operation 640 of FIG. 6A.

[0245] According to one embodiment, the processor (410) may identify a first area within the second screen (1110) based on the gaze of the user (510). The processor (410) may identify a second area within the second screen (1110) with respect to a location where a light source (1115) is arranged. For example, the processor (410) may identify that the first area corresponds to the second area. The processor (410) may identify that the first area corresponds to the area (1111) and that the second area corresponds to the area (1111). For example, the light source (1115) may be displayed in the area (1111) within the second screen (1110). The gaze of the user (510) may be located in the area (1111) within the second screen (1110).

[0246] According to one embodiment, within the second screen (1110), a visual object (1112) may be displayed on an area (1111). The processor (410) may display the visual object (1112) on an area (1111) where the user's (510) gaze is located. For example, the visual object (1112) may indicate that the display mode of the display (420) has been changed. For example, when the visual object (1112) is displayed on an area (1111) where a light source (1115) is arranged, visibility may decrease. The processor (410) may increase the visibility of the visual object (1112) by changing the location of the visual object (1112).

[0247] According to an embodiment, the processor (410) may change the position of the visual object (1112) based on identifying that the value related to the brightness of the light source (1115) is outside the second range. For example, even if the processor (410) applies a filter for adjusting the brightness to an area corresponding to the light source (1115) as described in FIG. 10, the visibility of the visual object (1112) may not increase. For example, if the brightness of the light source (1115) is too bright, the brightness of the light source (1115) may fall outside the range that the display (420) can express. The processor (410) may change the position of the visual object (1112) based on identifying that the brightness of the light source (1115) is outside the range that the display (420) can express.

[0248] According to one embodiment, the processor (410) can change the second screen (1110) to the third screen (1120) based on whether a visual object (1112) is displayed on an area (1111) where a light source (1115) is placed.

[0249] For example, the processor (410) may change the second screen (1110) to the third screen (1130) based on changing the position of the visual object (1112). As an example, the processor (410) may identify that the visual object (1112) is displayed on an area (1111) where a light source (1115) is arranged. The processor (410) may identify at least one light source within the second screen (1110) that is distinct from the light source (1115). The processor (410) may change the position of the visual object (1112) to an area that is distinct from the area where the at least one light source is displayed. The processor (410) may display an element (1113) indicating that the visual object (1112) of the visual object (1112) has been moved together with the visual object (1112) within the third screen (1120).

[0250] In FIG. 11, the shape of the element (1113) is illustrated as an arrow shape, but is not limited thereto. For example, the element (1113) may be configured to display an animation effect of an object (1112) moving. For example, the element (1113) may be configured to display a shadow effect displayed at the position before the object (1112) is moved.

[0251] For example, the processor (410) can identify an area within the second screen (1110) where the visual object (1112) has the best visibility. The processor (410) can change the position of the visual object (1112) to the identified area. For example, the processor (410) can identify the color and / or brightness of the visual object (1112). The processor (410) can identify an area where the visual object (1112) is to be displayed based on the color and / or brightness of objects within the second virtual space. The processor (410) can change the second screen (1110) to a third screen (1120) based on changing the position of the visual object (1112) to the identified area.

[0252] According to an embodiment, the processor (410) may apply a filter for adjusting a light source in FIG. 10 and change the position of a visual object (1112). For example, the filter applied for adjusting a light source and / or the position at which the visual object (1112) is changed may be set based on an application running on the wearable device (200) and / or a service provided by the wearable device (200). For example, the filter applied for adjusting a light source and / or the position at which the visual object (1112) is changed may be set based on a user's preference.

[0253] The above-described embodiment has been described as an example in which the position of the visual object (1112) is changed to improve the visibility of the visual object (1112), but is not limited thereto. For example, the processor (410) may display the visual object (1112) in another area to move the user's (510) gaze from the area (1111) to another area.

[0254] FIG. 12 illustrates an example of operation of a wearable device according to one embodiment.

[0255] Referring to FIG. 12, the processor (410) can identify that the display mode of the display (420) has changed from the first display mode to the second display mode. Based on identifying that the display mode of the display (420) has changed from the first display mode to the second display mode, the processor (410) can identify a second screen (1210) regarding a second virtual space according to the second display mode. For example, the processor (410) can identify the second screen (1210) based on operation 640 of FIG. 6A.

[0256] According to one embodiment, the processor (410) can identify that an area in the second screen (1210) where a visual object (1212) is displayed corresponds to an area in which an object (1211) is displayed. The processor (410) can identify information about the object (1211). For example, the processor (410) can identify a color of the object (1211). The processor (410) can identify that the color of the object (1211) corresponds to the color of the visual object (1212).

[0257] For example, if the color of the object (1211) corresponds to the color of the visual object (1212), the visibility of the visual object (1212) may be reduced. The processor (410) may change the characteristics of the visual object (1212). As an example, the processor (410) may change the color of the visual object (1212).

[0258] According to one embodiment, the processor (410) can change the second screen (1210) to the third screen (1220) based on changing the characteristics of the visual object (1212). For example, the processor (410) can change the second screen (1210) to the third screen (1220) based on changing the color of the visual object (1212). The processor (410) can improve the visibility of the visual object (1212) based on changing the color of the visual object (1212).

[0259] In FIG. 12, an example is shown in which the color of the visual object (1212) is changed when the color of the object (1211) corresponds to the color of the visual object (1212), but the present invention is not limited thereto. When the color of the object (1211) corresponds to the color of the visual object (1212), the processor (410) may change at least one of the shape, size, and / or transparency of the visual object (1212).

[0260] FIG. 13 illustrates an example of operation of a wearable device according to one embodiment.

[0261] Referring to FIG. 13, the processor (410) can identify that the display mode of the display (420) has changed from the first display mode to the second display mode. Based on identifying that the display mode of the display (420) has changed from the first display mode to the second display mode, the processor (410) can identify a second screen (1310) related to the second virtual space according to the second display mode. For example, the second screen (1310) can correspond to the second screen (1210) of FIG. 12.

[0262] In one embodiment, when the color of the object (1311) corresponds to the color of the visual object (1312), the visibility of the visual object (1312) may be reduced. The processor (410) may change the position of the visual object (1312).

[0263] For example, the processor (410) can identify a location within the second screen (1310) where the visual object (1312) is to be moved based on the color of the visual object (1312). The processor (410) can move the visual object (1312) to the identified location.

[0264] According to one embodiment, the processor (410) can change the second screen (1310) to the third screen (1320) based on changing the position of the visual object (1312). The processor (410) can improve the visibility of the visual object (1312) based on changing the position of the visual object (1312).

[0265] According to one embodiment, a wearable device may include a display, a camera, at least one sensor, a memory storing instructions and including one or more storage media, and at least one processor. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display a first screen regarding a first virtual space through the display while a display mode of the display is a first display mode. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify information regarding an external environment in which the wearable device is located while at least one screen including the first screen regarding the first virtual space is displayed. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify that the display mode of the display is changed from the first display mode to a second display mode after the information about the external environment is identified. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify a second screen relating to a second virtual space according to the second display mode based on identifying that the display mode of the display is changed from the first display mode to the second display mode.The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display a third screen changed from the second screen through the display while the display mode of the display is the second display mode, based on the first area within the second screen identified based on the information about the external environment and the gaze of the user of the wearable device.

[0266] According to one embodiment, the information about the external environment may include information about the location of a light source within the space where the wearable device is located.

[0267] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify information regarding brightness of one or more images of the external environment acquired through the camera. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify information regarding a location of the light source within the space in which the wearable device is located, based on the information regarding brightness of the one or more images of the external environment.

[0268] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify that the first region within the second screen, identified according to the user's gaze, corresponds to a second region within the second screen relative to a location where the light source is disposed. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to change data for the first region within the second screen based on identifying that the first region corresponds to the second region. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify the third screen changed from the second screen based on changing the data for the first region within the second screen.

[0269] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display the second screen through the display while the display mode of the display is the second display mode, based on identifying that the first region is distinct from the second region.

[0270] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify information about the brightness of the light source based on identifying that the first area corresponds to the second area. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to change a position of a visual object displayed in the first area within the second screen based on the information about the brightness of the light source. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify a third screen changed from the second screen based on changing the position of the visual object displayed in the first area within the second screen.

[0271] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display an element within the third screen indicating that the visual object has moved.

[0272] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify another location within the second screen, distinct from the location at which the light source is disposed. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to change at least one of a location of the visual object or a feature of the visual object based on information about an object displayed at the other location.

[0273] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify, using the at least one sensor, a value relating to movement of the wearable device. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to change the display mode of the display from the first display mode to the second display mode based on the value relating to the movement of the wearable device exceeding a designated value.

[0274] According to one embodiment, the first virtual space according to the first display mode may be configured to be distinct from the space in which the wearable device is located. The second virtual space according to the second display mode may be configured to include the space and one or more visual objects displayed in an overlapping manner in the space.

[0275] According to one embodiment, a method of a wearable device may include an operation of displaying a first screen regarding a first virtual space through the display while the display mode of the display is a first display mode. The method may include an operation of identifying information regarding an external environment in which the wearable device is located while at least one screen including the first screen regarding the first virtual space is displayed. The method may include an operation of identifying, after the information regarding the external environment is identified, that the display mode of the display is changed from the first display mode to a second display mode. The method may include an operation of identifying a second screen regarding a second virtual space according to the second display mode based on identifying that the display mode of the display is changed from the first display mode to the second display mode. The method may include an operation of displaying a third screen changed from the second screen through the display while the display mode of the display is the second display mode, based on the information about the external environment and the first area within the second screen identified according to the gaze of the user of the wearable device.

[0276] According to one embodiment, the information about the external environment may include information about the location of a light source within the space where the wearable device is located.

[0277] According to one embodiment, the method may include an operation of identifying information regarding the brightness of one or more images of the external environment acquired through the camera. The method may include an operation of identifying information regarding the location of the light source within the space where the wearable device is located, based on the information regarding the brightness of the one or more images of the external environment.

[0278] According to one embodiment, the method may include an operation of identifying that the first region within the second screen, identified based on the user's gaze, corresponds to a second region within the second screen with respect to a location where the light source is arranged. The method may include an operation of changing data for the first region within the second screen based on identifying that the first region corresponds to the second region. The method may include an operation of identifying the third screen changed from the second screen based on changing the data for the first region within the second screen.

[0279] According to one embodiment, the method may include an operation of displaying the second screen through the display while the display mode of the display is the second display mode, based on identifying that the first region is distinct from the second region.

[0280] According to one embodiment, the method may include an operation of identifying information about the brightness of the light source based on identifying that the first region corresponds to the second region. The method may include an operation of changing a position of a visual object displayed in the first region within the second screen based on the information about the brightness of the light source. The method may include an operation of identifying a third screen changed from the second screen based on changing the position of the visual object displayed in the first region within the second screen.

[0281] In one embodiment, the method may include an action of displaying an element within the third screen indicating that the visual object has moved.

[0282] In one embodiment, the method may include an operation of identifying a different location within the second screen, distinct from the location where the light source is positioned. The method may include an operation of changing at least one of the location of the visual object or a feature of the visual object based on information about the object displayed at the different location.

[0283] According to one embodiment, the method may include an operation of identifying a value related to movement of the wearable device using at least one sensor of the wearable device. The method may include an operation of changing the display mode of the display from the first display mode to the second display mode based on the value related to the movement of the wearable device exceeding a specified value.

[0284] According to one embodiment, a non-transitory computer readable storage medium may store one or more programs. The one or more programs may include instructions that, when executed by at least one processor of a wearable device having a display, a camera, and at least one sensor, cause the wearable device to display a first screen regarding a first virtual space through the display while the display mode of the display is a first display mode. The one or more programs may include instructions that cause the wearable device to identify information about an external environment while at least one screen including the first screen regarding the first virtual space is displayed. The one or more programs may include instructions that, after the information about the external environment is identified, cause the wearable device to identify that the display mode of the display is changed from the first display mode to a second display mode. The one or more programs may include instructions that cause the wearable device to identify a second screen related to a second virtual space according to the second display mode based on identifying that the display mode of the display has changed from the first display mode to the second display mode. The one or more programs may include instructions that cause the wearable device to display, through the display, a third screen changed from the second screen based on a first area within the second screen identified according to the information about the external environment and the gaze of the user of the wearable device while the display mode of the display is the second display mode.

[0285] According to the above-described embodiment, a wearable device (e.g., wearable device (200)) can switch the display mode of the display from AR mode to VR mode. The wearable device (e.g., wearable device (200)) can switch the display mode of the display from VR mode to AR mode. When the display mode of the display is switched, the illuminance perceived by the user may change depending on the brightness of the content and / or the external environment. The wearable device can change the screen to reduce the change in illuminance perceived by the user. As the wearable device changes the screen, the user can continue to gaze at the content even when the display mode of the display is switched.

[0286] Electronic devices according to embodiments disclosed herein may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to embodiments disclosed herein are not limited to the aforementioned devices.

[0287] The embodiments of this document and the terminology used herein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among the phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0288] In one embodiment of this document, the term "module" used may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0289] One embodiment of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0290] According to one embodiment, the method according to one embodiment disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0291] According to one embodiment, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and arranged in other components. According to one embodiment, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to one embodiment, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In wearable devices, display; camera; At least one sensor; A memory storing instructions and including one or more storage media; and At least one processor including a processing circuit, wherein the instructions, when individually or collectively executed by the at least one processor, cause the wearable device to: While the display mode of the above display is the first display mode, a first screen regarding the first virtual space is displayed through the display, While at least one screen including a first screen regarding the first virtual space is displayed, information about the external environment in which the wearable device is located is identified, After the information about the external environment is identified, it is identified that the display mode of the display is changed from the first display mode to the second display mode, Identifying a second screen for a second virtual space according to the second display mode based on identifying that the display mode of the display has changed from the first display mode to the second display mode, Based on the information about the external environment and the first area within the second screen identified according to the gaze of the user of the wearable device, causing a third screen changed from the second screen to be displayed through the display while the display mode of the display is the second display mode. Wearable devices.

2. In paragraph 1, information about the external environment is: Including information about the location of the light source within the space where the wearable device is located, Wearable devices.

3. In the second paragraph, when the instructions are individually or collectively executed by the at least one processor, the wearable device, Identifying information about brightness of one or more images of the external environment acquired through the camera, Based on information about the brightness of said one or more images of said external environment, causing said information about a location where said light source is arranged within said space where said wearable device is located to be identified. Wearable devices.

4. In the second paragraph, when the instructions are individually or collectively executed by the at least one processor, the wearable device, Identifying that the first area within the second screen, identified according to the user's gaze, corresponds to the second area within the second screen, with respect to the location where the light source is placed; Based on identifying that the first area corresponds to the second area, changing data for the first area within the second screen; Further set to identify the third screen changed from the second screen based on changing the data for the first area within the second screen, The first area within the second screen is, Based on identifying that the value of brightness within the first region is greater than or equal to a specified value, Wearable devices.

5. In the fourth paragraph, when the instructions are individually or collectively executed by the at least one processor, the wearable device, Based on identifying that the first region is distinct from the second region, causing the second screen to be displayed through the display while the display mode of the display is the second display mode. Wearable devices.

6. In the fourth paragraph, when the instructions are individually or collectively executed by the at least one processor, the wearable device, Based on identifying that the first region corresponds to the second region, information about the brightness of the light source is identified, Based on information about the brightness of the light source, changing the position of a visual object displayed in the first area within the second screen, Causing to identify the third screen changed from the second screen based on changing the position of the visual object displayed in the first area within the second screen; Wearable devices.

7. In the sixth paragraph, when the instructions are individually or collectively executed by the at least one processor, the wearable device, causing an element to be displayed within the third screen indicating that the visual object has moved; Wearable devices.

8. In the sixth paragraph, when the instructions are individually or collectively executed by the at least one processor, the wearable device, Within the second screen, identify another location that is distinct from the location where the light source is placed, Causing to change at least one of the location of the visual object or a feature of the visual object based on information about the object displayed at the other location. Wearable devices.

9. In the first paragraph, when the instructions are individually or collectively executed by the at least one processor, the wearable device, Using at least one sensor, a value related to movement of the wearable device is identified, Causing the display mode of the display to be changed from the first display mode to the second display mode based on the value relating to the movement of the wearable device exceeding a specified value; Wearable devices.

10. In the first paragraph, the first virtual space according to the first display mode is configured to be distinguished from the space in which the wearable device is located; The second virtual space according to the second display mode is, configured to include the above space and one or more visual objects that are displayed overlapping the above space, Wearable devices.

11. In the method of a wearable device, An operation of displaying a first screen regarding a first virtual space through the display while the display mode of the display of the wearable device is a first display mode; An operation of identifying information about an external environment in which the wearable device is located while at least one screen including a first screen relating to the first virtual space is displayed; An action of identifying that the display mode of the display is changed from the first display mode to the second display mode after the information about the external environment is identified; An operation of identifying a second screen regarding a second virtual space according to the second display mode based on identifying that the display mode of the display has changed from the first display mode to the second display mode; and An operation of displaying a third screen changed from the second screen through the display while the display mode of the display is the second display mode, based on the information about the external environment and the first area within the second screen identified according to the gaze of the user of the wearable device, method.

12. In clause 11, the information about the external environment is: Including information about the location of the light source within the space where the wearable device is located, method.

13. In the 12th paragraph, the method, An operation of identifying information about brightness of one or more images of the external environment acquired through a camera of the wearable device; and Further comprising an action of identifying information about a location where the light source is arranged within the space where the wearable device is located, based on information about brightness of the one or more images of the external environment. method.

14. In the 12th paragraph, the method, An action of identifying that the first area within the second screen, identified according to the user's gaze, corresponds to a second area within the second screen, with respect to a position where the light source is placed; An operation of changing data for the first area within the second screen based on identifying that the first area corresponds to the second area; and Further comprising an operation of identifying the third screen changed from the second screen based on changing the data for the first area within the second screen. method.

15. A non-transitory computer readable storage medium storing one or more programs, wherein the one or more programs, when executed by at least one processor of a wearable device having a display, a camera, and at least one sensor, While the display mode of the above display is the first display mode, a first screen regarding the first virtual space is displayed through the display, While at least one screen including a first screen relating to the first virtual space is displayed, information about the external environment is identified, After the information about the external environment is identified, it is identified that the display mode of the display is changed from the first display mode to the second display mode, Identifying a second screen for a second virtual space according to the second display mode based on identifying that the display mode of the display has changed from the first display mode to the second display mode, Instructions for causing the wearable device to display a third screen changed from the second screen through the display while the display mode of the display is the second display mode, based on the first area within the second screen identified according to the information about the external environment and the gaze of the user of the wearable device. Computer readable storage medium.

Images