Electronic device, method, and computer-readable storage medium for displaying visual object related to application in virtual space

Abstract

According to an embodiment, a wearable device may comprise: a display; at least one sensor; a memory for storing instructions; and at least one processor. The at least one processor may be configured to, when the instructions are executed: identify at least one application being executed in a first virtual space, while the first virtual space is displayed; switch the first virtual space to a second virtual space on the basis of the execution of a first application; identify a second application executed in the first virtual space among the at least one application, while the second virtual space is displayed; identify a motion of the wearable device while displaying a first portion of the second virtual space on the basis of a first direction in which a user's gaze is directed; and display a visual object related to the second application in a second portion of the second virtual space on the basis of a second direction in which the user's gaze is directed, the second direction being changed from the first direction according to the motion of the wearable device.

Description

Electronic device, method, and computer-readable storage medium for displaying visual objects related to an application within a virtual space

[0001] The following descriptions relate to electronic devices, methods, and computer-readable storage media for displaying visual objects relating to an application within a virtual space.

[0002] To provide an enhanced user experience, electronic devices are being developed that provide augmented reality (AR) and / or virtual reality (VR) services that display computer-generated information in conjunction with external objects in the real world. The electronic devices may be wearable devices that can be worn by a user. For example, the electronic devices may be AR glasses and / or head-mounted devices (HMDs).

[0003] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art related to the present disclosure.

[0004] According to one embodiment, a wearable device may include a display, at least one sensor, a memory storing instructions, and at least one processor. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify at least one application running within a first virtual space while the first virtual space is displayed. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify an input for running a first application within the first virtual space. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to switch the first virtual space to a second virtual space provided according to the first application based on the execution of the first application. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify, while the second virtual space is displayed, a second application running within the first virtual space, among the at least one application. The at least one processor, when the instructions are executed, may cause the wearable device to identify a motion of the wearable device while displaying a first portion of the second virtual space based on a first direction in which the user's gaze is directed. The at least one processor, when the instructions are executed, may cause the wearable device to display a visual object relating to the second application within a second portion of the second virtual space based on a second direction in which the user's gaze is directed, which direction has changed from the first direction in response to the motion of the wearable device.

[0005] According to one embodiment, a method of a wearable device may include an operation of identifying at least one application running in a first virtual space while the first virtual space is displayed. The method may include an operation of identifying an input for running a first application within the first virtual space. The method may include an operation of switching the first virtual space to a second virtual space provided according to the first application based on the execution of the first application. The method may include an operation of identifying, while the second virtual space is displayed, a second application running within the first virtual space among the at least one application. The method may include an operation of identifying a motion of the wearable device while displaying a first portion of the second virtual space based on a first direction in which the user's gaze is directed. The method may include an operation of displaying a visual object related to the second application within a second portion of the second virtual space based on a second direction in which the user's gaze is directed, which direction is changed from the first direction according to the motion of the wearable device.

[0006] According to one embodiment, a non-transitory computer readable storage medium may store one or more programs. The one or more programs may include instructions that, when executed by at least one processor of a wearable device having a display and at least one sensor, cause the wearable device to identify at least one application running within a first virtual space while the first virtual space is displayed. The one or more programs may include instructions that, when executed by the at least one processor of the wearable device, cause the wearable device to identify an input for running a first application within the first virtual space. The one or more programs may include instructions that, when executed by the at least one processor of the wearable device, cause the wearable device to transition the first virtual space to a second virtual space provided according to the first application based on the execution of the first application. The one or more programs may include instructions that, when executed by the at least one processor of the wearable device, cause the wearable device to identify, while the second virtual space is displayed, a second application running within the first virtual space among the at least one application. The one or more programs may include instructions that, when executed by the at least one processor of the wearable device, cause the wearable device to identify a motion of the wearable device while displaying a first portion of the second virtual space based on a first direction toward which a user's gaze is directed.The one or more programs may include instructions that, when executed by the at least one processor of the wearable device, cause the wearable device to display a visual object related to the second application within a second portion of the second virtual space based on a second direction in which the gaze of the user is directed, which direction is changed from the first direction according to the motion of the wearable device.

[0007] FIG. 1 is a block diagram of an electronic device within a network environment according to one embodiment.

[0008] FIGS. 2A and 2B illustrate an example of a perspective view of a wearable device according to one embodiment.

[0009] FIGS. 3A and 3B illustrate an example of an appearance of a wearable device according to one embodiment.

[0010] FIG. 4 illustrates an example block diagram of a wearable device according to one embodiment.

[0011] FIG. 5A illustrates an example of a virtual space displayed through a display according to one embodiment.

[0012] FIG. 5b illustrates an example of a portion of a virtual space displayed according to the direction of a user's gaze, according to one embodiment.

[0013] FIG. 6 illustrates an example of a virtual space for executing at least one application, according to one embodiment.

[0014] FIG. 7 illustrates an example of a virtual space for providing an interface for an application, according to one embodiment.

[0015] FIG. 8a illustrates an example of a virtual space being converted, according to one embodiment.

[0016] FIG. 8b illustrates an example of a virtual space being switched, according to one embodiment.

[0017] Figure 9 illustrates a flowchart regarding the operation of a wearable device according to one embodiment.

[0018] FIG. 10A illustrates an example of operation of a wearable device for displaying a visual object related to a second application within a second virtual space, according to one embodiment.

[0019] FIG. 10b illustrates an example of operation of a wearable device for displaying a visual object related to a second application within a second virtual space, according to one embodiment.

[0020] FIG. 11 illustrates an example of operation of a wearable device for displaying a visual object related to a second application within a second virtual space, according to one embodiment.

[0021] FIG. 12A illustrates an example of operation of a wearable device for displaying a visual object related to a second application within a second virtual space, according to one embodiment.

[0022] FIG. 12b illustrates an example of operation of a wearable device for displaying a visual object related to a second application within a second virtual space, according to one embodiment.

[0023] FIG. 13 illustrates an example of operation of a wearable device for displaying a visual object related to a second application within a second virtual space, according to one embodiment.

[0024] FIG. 14 illustrates an example of operation of a wearable device for displaying a visual object related to a second application within a second virtual space, according to one embodiment.

[0025] FIG. 15 illustrates an example of operation of a wearable device for displaying an application running within a first virtual space, according to one embodiment.

[0026] FIG. 16 illustrates an example of operation of a wearable device for displaying an application running within a first virtual space, according to one embodiment.

[0027] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0028] FIG. 1 is a block diagram of an electronic device within a network environment according to one embodiment.

[0029] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0030] The processor (120) may control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) by executing, for example, software (e.g., a program (140)), and may perform various data processing or calculations. According to one embodiment, as at least a part of the data processing or calculation, the processor (120) may store a command or data received from another component (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the command or data stored in the volatile memory (132), and store the resulting data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or a secondary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together therewith. For example, if the electronic device (101) includes a main processor (121) and a secondary processor (123), the secondary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a specified function. The secondary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0031] The auxiliary processor (123) may control at least a part of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0032] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0033] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0034] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0035] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. According to one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0036] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0037] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0038] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0039] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0040] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0041] A haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0042] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0043] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0044] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0045] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0046] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) may support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0047] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. According to some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0048] In one embodiment, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0049] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0050] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service by itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that have received the request may execute at least a part of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101).

[0051] The electronic device (101) may process the result as is or additionally and provide it as at least part of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199). The electronic device (101) may be applied to intelligent services (e.g., smart homes, smart cities, smart cars, or healthcare) based on 5G communication technology and IoT-related technology.

[0052] FIGS. 2A and 2B illustrate an example of a perspective view of a wearable device according to one embodiment.

[0053] According to one embodiment, the wearable device (200) may have the form of glasses that are wearable on a body part of a user (e.g., head). The wearable device (200) of FIGS. 2A and 2B may be an example of the electronic device (101) of FIG. 1. The wearable device (200) may include a head-mounted display (HMD). For example, the housing of the wearable device (200) may include a flexible material, such as rubber and / or silicone, that is configured to fit closely to a portion of the user's head (e.g., a portion of the face surrounding both eyes). For example, the housing of the wearable device (200) may include one or more straps that are capable of being twined around the user's head, and / or one or more temples that are detachably attachable to the ears of the head.

[0054] Referring to FIG. 2A, according to one embodiment, a wearable device (200) may include at least one display (250) and a frame (295) supporting at least one display (250).

[0055] According to one embodiment, the wearable device (200) can be worn on a part of a user's body. The wearable device (200) can provide augmented reality (AR), virtual reality (VR), or mixed reality (MR) that combines augmented reality and virtual reality to the user wearing the wearable device (200). For example, the wearable device (200) can display a virtual reality image provided from at least one optical device (282, 284) of FIG. 2B on at least one display (250) in response to a user's designated gesture acquired through the motion recognition cameras (260-2, 260-3) of FIG. 2B.

[0056] According to one embodiment, at least one display (250) may provide visual information to a user. For example, at least one display (250) may include a transparent or translucent lens. At least one display (250) may include a first display (250-1) and / or a second display (250-2) spaced apart from the first display (250-1). For example, the first display (250-1) and the second display (250-2) may be positioned at positions corresponding to the user's left and right eyes, respectively.

[0057] Referring to FIG. 2B, at least one display (250) can provide the user with visual information transmitted from external light and other visual information distinct from the visual information through a lens included in the at least one display (250). The lens can be formed based on at least one of a Fresnel lens, a pancake lens, or a multi-channel lens. For example, the at least one display (250) can include a first surface (231) and a second surface (232) opposite to the first surface (231). A display area can be formed on the second surface (232) of the at least one display (250). When the user wears the wearable device (200), external light can be transmitted to the user by being incident on the first surface (231) and transmitted through the second surface (232). As another example, at least one display (250) can display an augmented reality image combined with a virtual reality image provided from at least one optical device (282, 284) on a real screen transmitted through external light, in a display area formed on the second surface (232).

[0058] In one embodiment, at least one display (250) may include at least one waveguide (233, 234) that diffracts light emitted from at least one optical device (282, 284) and transmits the diffracted light to a user. The at least one waveguide (233, 234) may be formed based on at least one of glass, plastic, or polymer. A nano-pattern may be formed on at least a portion of the exterior or interior of the at least one waveguide (233, 234). The nano-pattern may be formed based on a grating structure having a polygonal and / or curved shape. Light incident on one end of the at least one waveguide (233, 234) may be propagated to the other end of the at least one waveguide (233, 234) by the nano-pattern. At least one waveguide (233, 234) may include at least one diffractive element (e.g., a diffractive optical element (DOE), a holographic optical element (HOE)) and at least one reflective element (e.g., a reflective mirror). For example, at least one waveguide (233, 234) may be arranged within the wearable device (200) to guide a screen displayed by at least one display (250) to the user's eyes. For example, the screen may be transmitted to the user's eyes based on total internal reflection (TIR) ​​occurring within the at least one waveguide (233, 234).

[0059] The wearable device (200) can analyze an object included in a real image collected through a shooting camera (260-4), combine a virtual object corresponding to an object to be provided with augmented reality among the analyzed objects, and display the virtual object on at least one display (250). The virtual object can include at least one of text and an image regarding various information related to the object included in the real image. The wearable device (200) can analyze the object based on a multi-camera such as a stereo camera. For the object analysis, the wearable device (200) can perform spatial recognition (e.g., simultaneous localization and mapping (SLAM)) using a multi-camera and / or time-of-flight (ToF). A user wearing the wearable device (200) can view an image displayed on at least one display (250).

[0060] According to one embodiment, the frame (295) may be formed as a physical structure that allows the wearable device (200) to be worn on the user's body. According to one embodiment, the frame (295) may be configured so that, when the user wears the wearable device (200), the first display (250-1) and the second display (250-2) can be positioned corresponding to the user's left and right eyes. The frame (295) may support at least one display (250). For example, the frame (295) may support the first display (250-1) and the second display (250-2) to be positioned corresponding to the user's left and right eyes.

[0061] Referring to FIG. 2A, the frame (295) may include a region (220) that at least partially contacts a part of the user's body when the user wears the wearable device (200). For example, the region (220) of the frame (295) that contacts a part of the user's body may include a region that contacts a part of the user's nose, a part of the user's ear, and a part of the side of the user's face that the wearable device (200) makes contact with. According to one embodiment, the frame (295) may include a nose pad (210) that contacts a part of the user's body. When the wearable device (200) is worn by the user, the nose pad (210) may contact a part of the user's nose. The frame (295) may include a first temple (204) and a second temple (205) that contact a part of the user's body that is distinct from the part of the user's body.

[0062] For example, the frame (295) may include a first rim (201) that surrounds at least a portion of the first display (250-1), a second rim (202) that surrounds at least a portion of the second display (250-2), a bridge (203) that is disposed between the first rim (201) and the second rim (202), a first pad (211) that is disposed along a portion of the edge of the first rim (201) from one end of the bridge (203), a second pad (212) that is disposed along a portion of the edge of the second rim (202) from the other end of the bridge (203), a first temple (204) that extends from the first rim (201) and is fixed to a portion of an ear of the wearer, and a second temple (205) that extends from the second rim (202) and is fixed to a portion of an ear opposite the ear. The first pad (211) and the second pad (212) may be in contact with a portion of the user's nose, and the first temple (204) and the second temple (205) may be in contact with a portion of the user's face and a portion of the user's ear. The temples (204, 205) may be rotatably connected to the rim through the hinge units (206, 207) of FIG. 2B. The first temple (204) may be rotatably connected to the first rim (201) through the first hinge unit (206) disposed between the first rim (201) and the first temple (204). The second temple (205) may be rotatably connected to the second rim (202) through the second hinge unit (207) disposed between the second rim (202) and the second temple (205). According to one embodiment, the wearable device (200) can identify an external object (e.g., a user's fingertip) touching the frame (295) and / or a gesture performed by the external object by using a touch sensor, a grip sensor, and / or a proximity sensor formed on at least a portion of a surface of the frame (295).

[0063] According to one embodiment, the wearable device (200) may include hardwares that perform various functions (e.g., hardwares to be described later based on the block diagram of FIG. 4). For example, the hardwares may include a battery module (270), an antenna module (275), at least one optical device (282, 284), speakers (e.g., speakers 255-1, 255-2), a microphone (e.g., microphones 265-1, 265-2, 265-3), a light-emitting module (not shown), and / or a printed circuit board (PCB) (290) (e.g., a printed circuit board). The various hardwares may be arranged within a frame (295).

[0064] According to one embodiment, the microphones (e.g., microphones 265-1, 265-2, 265-3) of the wearable device (200) may be disposed on at least a portion of the frame (295) to acquire sound signals. A first microphone (265-1) disposed on the bridge (203), a second microphone (265-2) disposed on the second rim (202), and a third microphone (265-3) disposed on the first rim (201) are illustrated in FIG. 2B , but the number and arrangement of the microphones (265) are not limited to the embodiment of FIG. 2B . When the number of microphones (265) included in the wearable device (200) is two or more, the wearable device (200) may identify the direction of the sound signal by using a plurality of microphones disposed on different portions of the frame (295).

[0065] In one embodiment, at least one optical device (282, 284) can project a virtual object onto at least one display (250) to provide various image information to a user. For example, at least one optical device (282, 284) can be a projector. At least one optical device (282, 284) can be disposed adjacent to at least one display (250) or can be included within at least one display (250) as a part of at least one display (250). In one embodiment, the wearable device (200) can include a first optical device (282) corresponding to a first display (250-1) and a second optical device (284) corresponding to a second display (250-2). For example, at least one optical device (282, 284) may include a first optical device (282) disposed at an edge of a first display (250-1) and a second optical device (284) disposed at an edge of a second display (250-2). The first optical device (282) may transmit light to a first waveguide (233) disposed on the first display (250-1), and the second optical device (284) may transmit light to a second waveguide (234) disposed on the second display (250-2).

[0066] In one embodiment, the camera (260) may include a recording camera (260-4), an eye tracking camera (ET CAM) (260-1), and / or motion recognition cameras (260-2, 260-3). The recording camera (260-4), the eye tracking camera (260-1), and the motion recognition cameras (260-2, 260-3) may be positioned at different locations on the frame (295) and may perform different functions. The eye tracking camera (260-1) may output data indicating the position or gaze of the eyes of a user wearing the wearable device (200). For example, the wearable device (200) may detect the gaze from an image including the user's pupils obtained through the eye tracking camera (260-1). The wearable device (200) can identify an object (e.g., a real object and / or a virtual object) focused on by the user using the user's gaze acquired through the gaze tracking camera (260-1). The wearable device (200) that has identified the focused object can execute a function (e.g., gaze interaction) for interaction between the user and the focused object. The wearable device (200) can express a part corresponding to the eye of an avatar representing the user in a virtual space using the user's gaze acquired through the gaze tracking camera (260-1). The wearable device (200) can render an image (or screen) displayed on at least one display (250) based on the position of the user's eyes. For example, the visual quality of a first region related to the gaze within the image and the visual quality (e.g., resolution, brightness, saturation, grayscale, PPI) of a second region distinguished from the first region may be different from each other. The wearable device (200) can obtain an image having visual quality of a first area matching the user's gaze and visual quality of a second area using foveated rendering.For example, if the wearable device (200) supports an iris recognition function, user authentication can be performed based on iris information acquired using the gaze tracking camera (260-1). An example in which the gaze tracking camera (260-1) is positioned toward the user's right eye is illustrated in FIG. 2B, but the embodiment is not limited thereto, and the gaze tracking camera (260-1) may be positioned solely toward the user's left eye, or toward both eyes.

[0067] In one embodiment, the capturing camera (260-4) can capture an actual image or background to be aligned with a virtual image to implement augmented reality or mixed reality content. The capturing camera (260-4) can be used to acquire a high-resolution image based on HR (high resolution) or PV (photo video). The capturing camera (260-4) can capture an image of a specific object existing at a location viewed by the user and provide the image to at least one display (250). The at least one display (250) can display a single image in which information about an actual image or background including the image of the specific object acquired using the capturing camera (260-4) is superimposed on a virtual image provided through at least one optical device (282, 284). The wearable device (200) can compensate for depth information (e.g., the distance between the wearable device (200) and an external object acquired through a depth sensor) using the image acquired through the capturing camera (260-4). The wearable device (200) can perform object recognition through an image acquired using a photographing camera (260-4). The wearable device (200) can perform a function of focusing on an object (or subject) in an image (e.g., auto focus) and / or an optical image stabilization (OIS) function (e.g., anti-shake function) using the photographing camera (260-4). The wearable device (200) can perform a pass-through function to display an image acquired through the photographing camera (260-4) by overlapping at least a portion of a screen representing a virtual space on at least one display (250). In one embodiment, the photographing camera (260-4) can be disposed on a bridge (203) disposed between a first rim (201) and a second rim (202).

[0068] The gaze tracking camera (260-1) can implement more realistic augmented reality by tracking the gaze of a user wearing the wearable device (200) and matching the user's gaze with visual information provided to at least one display (250). For example, when the wearable device (200) looks straight ahead, the wearable device (200) can naturally display environmental information related to the user's front at a location where the user is located on at least one display (250). The gaze tracking camera (260-1) can be configured to capture an image of the user's pupil to determine the user's gaze. For example, the gaze tracking camera (260-1) can receive gaze detection light reflected from the user's pupil and track the user's gaze based on the position and movement of the received gaze detection light. In one embodiment, the gaze tracking camera (260-1) can be positioned at positions corresponding to the user's left and right eyes. For example, the gaze tracking camera (260-1) may be positioned within the first rim (201) and / or the second rim (202) to face the direction in which the user wearing the wearable device (200) is positioned.

[0069] The gesture recognition camera (260-2, 260-3) can recognize the movement of the user's entire body, such as the user's torso, hand, or face, or a part of the body, and thereby provide a specific event on a screen provided on at least one display (250). The gesture recognition camera (260-2, 260-3) can recognize the user's gesture (gesture recognition), obtain a signal corresponding to the gesture, and provide a display corresponding to the signal on at least one display (250). The processor can identify the signal corresponding to the gesture, and perform a designated function based on the identification. The gesture recognition camera (260-2, 260-3) can be used to perform simultaneous localization and mapping (SLAM) for 6 degrees of freedom pose (6 dof pose) and / or a spatial recognition function using a depth map. The processor may perform gesture recognition and / or object tracking functions using the motion recognition cameras (260-2, 260-3). In one embodiment, the motion recognition cameras (260-2, 260-3) may be positioned on the first rim (201) and / or the second rim (202).

[0070] The camera (260) included in the wearable device (200) is not limited to the above-described gaze tracking camera (260-1) and motion recognition cameras (260-2, 260-3). For example, the wearable device (200) can identify an external object included in the FoV using a camera positioned toward the user's FoV. The wearable device (200) can identify an external object based on a sensor for identifying the distance between the wearable device (200) and the external object, such as a depth sensor and / or a time of flight (ToF) sensor. The camera (260) positioned toward the FoV can support an autofocus function and / or an optical image stabilization (OIS) function. For example, the wearable device (200) may include a camera (260) (e.g., a face tracking (FT) camera) positioned toward the face to obtain an image including the face of a user wearing the wearable device (200).

[0071] Although not shown, in one embodiment, the wearable device (200) may further include a light source (e.g., an LED) that emits light toward a subject (e.g., a user's eyes, face, and / or an external object within the FoV) being photographed using the camera (260). The light source may include an infrared wavelength LED. The light source may be disposed on at least one of the frame (295) and the hinge units (206, 207).

[0072] According to one embodiment, the battery module (270) may supply power to electronic components of the wearable device (200). In one embodiment, the battery module (270) may be disposed within the first temple (204) and / or the second temple (205). For example, the battery module (270) may be a plurality of battery modules (270). The plurality of battery modules (270) may be disposed within each of the first temple (204) and the second temple (205). In one embodiment, the battery module (270) may be disposed at an end of the first temple (204) and / or the second temple (205).

[0073] The antenna module (275) can transmit signals or power to the outside of the wearable device (200), or receive signals or power from the outside. In one embodiment, the antenna module (275) can be positioned within the first temple (204) and / or the second temple (205). For example, the antenna module (275) can be positioned close to one surface of the first temple (204) and / or the second temple (205).

[0074] The speaker (255) can output an audio signal to the outside of the wearable device (200). The audio output module may be referred to as a speaker. In one embodiment, the speaker (255) may be positioned within the first temple (204) and / or the second temple (205) so as to be positioned adjacent to the ear of a user wearing the wearable device (200). For example, the speaker (255) may include a second speaker (255-2) positioned within the first temple (204) and thus adjacent to the user's left ear, and a first speaker (255-1) positioned within the second temple (205) and thus adjacent to the user's right ear.

[0075] The light-emitting module (not shown) may include at least one light-emitting element. The light-emitting module may emit light of a color corresponding to a specific state or emit light with an action corresponding to a specific state in order to visually provide information regarding a specific state of the wearable device (200) to the user. For example, when the wearable device (200) requires charging, it may emit red light at a regular cycle. In one embodiment, the light-emitting module may be disposed on the first rim (201) and / or the second rim (202).

[0076] Referring to FIG. 2B, according to one embodiment, a wearable device (200) may include a printed circuit board (PCB) (290). The PCB (290) may be included in at least one of the first temple (204) or the second temple (205). The PCB (290) may include an interposer disposed between at least two sub-PCBs. One or more hardwares included in the wearable device (200) (e.g., hardwares illustrated by different blocks in FIG. 4) may be disposed on the PCB (290). The wearable device (200) may include a flexible PCB (FPCB) for interconnecting the hardwares.

[0077] According to one embodiment, a wearable device (200) may include at least one of a gyro sensor, a gravity sensor, and / or an acceleration sensor for detecting a posture of the wearable device (200) and / or a posture of a body part (e.g., a head) of a user wearing the wearable device (200). Each of the gravity sensor and the acceleration sensor may measure gravitational acceleration and / or acceleration based on mutually perpendicular designated three-dimensional axes (e.g., the x-axis, the y-axis, and the z-axis). The gyro sensor may measure an angular velocity of each of the designated three-dimensional axes (e.g., the x-axis, the y-axis, and the z-axis). At least one of the gravity sensor, the acceleration sensor, and the gyro sensor may be referred to as an inertial measurement unit (IMU). According to one embodiment, the wearable device (200) may identify a user's motion and / or gesture performed to execute or terminate a specific function of the wearable device (200) based on the IMU.

[0078] Figures 3a and 3b illustrate an example of an exterior appearance of a wearable device according to one embodiment. The wearable device (300) of Figures 3a and 3b may be an example of the electronic device (101) of Figure 1 and the wearable device (200) of Figures 2a and 2b. An example of an exterior appearance of a first side (310) of a housing of the wearable device (200) according to one embodiment is illustrated in Figure 3a, and an example of an exterior appearance of a second side (320) opposite to the first side (310) may be illustrated in Figure 3b.

[0079] Referring to FIG. 3A, according to one embodiment, a first surface (310) of a wearable device (200) may have a form attachable to a body part of a user (e.g., the face of the user). Although not shown, the wearable device (200) may further include a strap for fixing to a body part of a user, and / or one or more temples (e.g., the first temple (204) and / or the second temple (205) of FIGS. 2A and 2B). A first display (250-1) for outputting an image to a left eye among the user's two eyes, and a second display (250-2) for outputting an image to a right eye among the user's two eyes may be disposed on the first surface (310). The wearable device (200) is formed on the first surface (310) and may further include a rubber or silicone packing to prevent interference by light (e.g., ambient light) different from the light emitted from the first display (250-1) and the second display (250-2).

[0080] According to one embodiment, the wearable device (200) may include cameras (260-1) for photographing and / or tracking both eyes of the user adjacent to each of the first display (250-1) and the second display (250-2). The cameras (260-1) may be referred to as the gaze tracking camera (260-1) of FIG. 2B. According to one embodiment, the wearable device (200) may include cameras (260-5, 260-6) for photographing and / or recognizing the face of the user. The cameras (260-5, 260-6) may be referred to as FT cameras. The wearable device (200) may control an avatar representing the user in a virtual space based on the motion of the user's face identified using the cameras (260-5, 260-6). For example, the wearable device (200) may change the texture and / or shape of a portion of an avatar (e.g., a portion of an avatar representing a human face) using information obtained by cameras (260-5, 260-6) (e.g., FT cameras) and representing the facial expression of a user wearing the wearable device (200).

[0081] Referring to FIG. 3B, a camera (e.g., cameras 260-7, 260-8, 260-9, 260-10, 260-11, 260-12)) and / or a sensor (e.g., a depth sensor 330) may be disposed on a second surface (320) opposite to the first surface (310) of FIG. 3A to obtain information related to the external environment of the wearable device (200). For example, the cameras (260-7, 260-8, 260-9, 260-10) may be disposed on the second surface (320) to recognize external objects. Cameras (260-7, 260-8, 260-9, 260-10) may be referenced to the motion recognition cameras (260-2, 260-3) of FIG. 2B.

[0082] For example, using cameras (260-11, 260-12), the wearable device (200) can obtain images and / or videos to be transmitted to each of the user's eyes. The camera (260-11) can be placed on the second face (320) of the wearable device (200) to obtain an image to be displayed through the second display (250-2) corresponding to the right eye among the two eyes. The camera (260-12) can be placed on the second face (320) of the wearable device (200) to obtain an image to be displayed through the first display (250-1) corresponding to the left eye among the two eyes. The cameras (260-11, 260-12) can be referred to as the shooting camera (260-4) of FIG. 2B.

[0083] According to one embodiment, the wearable device (200) may include a depth sensor (330) disposed on the second face (320) to identify a distance between the wearable device (200) and an external object. Using the depth sensor (330), the wearable device (200) may obtain spatial information (e.g., a depth map) for at least a portion of the FoV of a user wearing the wearable device (200). Although not illustrated, a microphone may be disposed on the second face (320) of the wearable device (200) to obtain sound output from an external object. The number of microphones may be one or more, depending on the embodiment.

[0084] FIG. 4 illustrates an example block diagram of a wearable device according to one embodiment. Referring to FIG. 4, a wearable device (200) according to one embodiment may include at least one of a processor (410), a memory (415), a display (420), a camera (425), a sensor (430), or a communication circuit (435). The processor (410), the memory (415), the display (420), the camera (425), the sensor (430), and the communication circuit (435) may be electrically and / or operably coupled with each other by an electronic component such as a communication bus (402). The type and / or number of hardware components included in the wearable device (200) are not limited to those illustrated in FIG. 4. For example, the wearable device (200) may include only some of the hardware components illustrated in FIG. 4. The elements within the memory described below (e.g., layers and / or modules) may be logically distinct, but are not limited thereto.

[0085] According to one embodiment, the processor (410) of the wearable device (200) may include a hardware component for processing data based on one or more instructions. The hardware component for processing data may include, for example, an arithmetic and logic unit (ALU), a field programmable gate array (FPGA), and / or a central processing unit (CPU). The number of processors (410) may be one or more. For example, the processor (410) may have a multi-core processor structure such as a dual core, a quad core, or a hexa core.

[0086] The memory (415) of the wearable device (200) according to one embodiment may include a hardware component for storing data and / or instructions input and / or output to the processor (410). The memory (415) may include, for example, a volatile memory such as a random-access memory (RAM) and / or a non-volatile memory such as a read-only memory (ROM). The volatile memory may include, for example, at least one of a dynamic RAM (DRAM), a static RAM (SRAM), a cache RAM, and a pseudo SRAM (PSRAM). The non-volatile memory may include, for example, at least one of a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), a flash memory, a hard disk, a compact disc, and an embedded multi media card (eMMC).

[0087] In one embodiment, the display (420) of the wearable device (200) can output visualized information to a user of the wearable device (200). For example, the display (420) can be controlled by a processor (410) including a circuit such as a graphic processing unit (GPU) to output visualized information to the user. The display (420) can include a flat panel display (FPD) and / or electronic paper. The FPD can include a liquid crystal display (LCD), a plasma display panel (PDP), and / or one or more light emitting diodes (LEDs). The LEDs can include organic LEDs (OLEDs).

[0088] In one embodiment, the camera (425) of the wearable device (200) may include one or more optical sensors (e.g., a charged coupled device (CCD) sensor, a complementary metal oxide semiconductor (CMOS) sensor) that generate electrical signals representing the color and / or brightness of light. The plurality of optical sensors included in the camera (425) may be arranged in the form of a two-dimensional array. The camera (425) may acquire electrical signals of each of the plurality of optical sensors substantially simultaneously to generate two-dimensional frame data corresponding to light reaching the optical sensors of the two-dimensional array. For example, photographic data captured using the camera (425) may mean one (a) two-dimensional frame data acquired from the camera (425). For example, video data captured using the camera (425) may mean a sequence of a plurality of two-dimensional frame data acquired from the camera (425) according to a frame rate. The camera (425) may further include a flash light that is positioned toward the direction in which the camera (425) receives light and outputs light toward the direction.

[0089] According to one embodiment, the wearable device (200) may include a plurality of cameras, for example, cameras (425), arranged in different directions. A first camera among the plurality of cameras may be referred to as a motion recognition camera (e.g., motion recognition cameras 260-2 and 260-3 of FIG. 2B ), and a second camera may be referred to as a gaze tracking camera (e.g., gaze tracking camera 260-1 of FIG. 2B ). The wearable device (200) may identify a position, shape, and / or gesture of a hand using an image acquired using the first camera. The wearable device (200) may identify a direction of a gaze of a user wearing the wearable device (200) using an image acquired using the second camera. For example, the direction in which the first camera faces may be opposite to the direction in which the second camera faces.

[0090] According to one embodiment, a sensor (430) of a wearable device (200) may generate electrical information that may be processed by a processor (410) and / or a memory (415) of the wearable device (200) from non-electronic information related to the wearable device (200). The information may be referred to as sensor data. The sensor (430) may include a global positioning system (GPS) sensor, an image sensor, an ambient light sensor, and / or a time-of-flight (ToF) sensor for detecting a geographic location of the wearable device (200), and an inertial measurement unit (IMU) for detecting a physical motion of the wearable device (200).

[0091] In one embodiment, the communication circuit (435) of the wearable device (200) may include hardware components for supporting transmission and / or reception of electrical signals between the wearable device (200) and an external electronic device. The communication circuit (435) may include, for example, at least one of a modem (MODEM), an antenna, and an optical / electronic (O / E) converter. The communication circuit (435) may support transmission and / or reception of electrical signals based on various types of protocols, such as Ethernet, a local area network (LAN), a wide area network (WAN), wireless fidelity (WiFi), Bluetooth, Bluetooth low energy (BLE), ZigBee, long term evolution (LTE), 5G NR (new radio), and / or 6G.

[0092] According to one embodiment, one or more instructions (or commands) representing operations and / or actions to be performed on data by the processor (410) of the wearable device (200) may be stored in the memory (415) of the wearable device (200). A set of one or more instructions may be referred to as firmware, an operating system, a process, a routine, a sub-routine, and / or an application. For example, the wearable device (200) and / or the processor (410) may perform at least one of the operations according to the embodiments described below when a set of a plurality of instructions distributed in the form of an operating system, firmware, a driver, and / or an application is executed. Hereinafter, the fact that an application is installed in a wearable device (200) may mean that one or more instructions provided in the form of an application are stored in a memory (415), and that the one or more applications are stored in a format executable by the processor (410) (e.g., a file having an extension specified by the operating system of the wearable device (200)). As an example, an application may include a program and / or a library related to a service provided to a user.

[0093] Referring to FIG. 4, programs installed in the wearable device (200) may be classified into one of different layers, including an application layer (440), a framework layer (450), and / or a hardware abstraction layer (HAL) (480), based on the target. For example, programs (e.g., modules or drivers) designed to target the hardware of the wearable device (200) (e.g., the display (420), the camera (425), and / or the sensor (430)) may be classified within the hardware abstraction layer (480). The framework layer (450) may be referred to as an XR framework layer in that it includes one or more programs for providing an XR (extended reality) service. For example, FIG. 4 illustrates layers being divided within the memory (415), the layers may be logically divided. However, the present invention is not limited thereto. Depending on the embodiment, the layers may be stored in a designated area within the memory (415).

[0094] For example, within the framework layer (450), programs designed to target at least one of the hardware abstraction layer (480) and / or the application layer (440) (e.g., a position tracker (471), a space recognizer (472), a gesture tracker (473), and / or an eye tracker (474), a face tracker (475)) may be classified. Programs classified within the framework layer (450) may provide an executable API (application programming interface) based on other programs.

[0095] For example, within the application layer (440), programs designed to target users controlling wearable devices (200) may be classified. Examples of programs classified into the application layer (440) include, but are not limited to, an XR (extended reality) system UI (user interface) and / or an XR application (442). For example, programs (e.g., software applications) classified into the application layer (440) may call an API (application programming interface) to cause execution of functions supported by programs classified into the framework layer (450).

[0096] For example, the wearable device (200) may display one or more visual objects on the display (420) for performing interaction with a user for using a virtual space based on the execution of the XR system UI (441). A visual object may refer to an object that can be deployed within a screen for transmitting and / or interacting with information, such as text, an image, an icon, a video, a button, a checkbox, a radio button, a text box, a slider, and / or a table. A visual object may be referred to as a visual guide, a virtual object, a visual element, a UI element, a view object, and / or a view element. The wearable device (200) may provide a service for controlling functions available within a virtual space to the user based on the execution of the XR system UI (441).

[0097] Referring to FIG. 4, a lightweight renderer (443) and / or an XR plug-in (444) are illustrated as being included within the XR system UI (441), but are not limited thereto. For example, the XR system UI (441) may cause execution of functions supported by the lightweight renderer (443) and / or the XR plug-in (444) included within the framework layer (450).

[0098] For example, the wearable device (200) may obtain resources (e.g., APIs, system processes, and / or libraries) used to define, create, and / or execute a rendering pipeline that allows partial changes based on the execution of a lightweight renderer (443). The lightweight renderer (443) may be referred to as a lightweight render pipeline in terms of defining a rendering pipeline that allows partial changes. The lightweight renderer (443) may include a renderer built prior to the execution of a software application (e.g., a prebuilt renderer). For example, the wearable device (200) may obtain resources (e.g., APIs, system processes, and / or libraries) used to define, create, and / or execute an entire rendering pipeline based on the execution of an XR plug-in (444). The XR plugin (444) may be referred to as an open XR native client from the perspective of defining (or configuring) the entire rendering pipeline.

[0099] For example, the wearable device (200) may display a screen representing at least a portion of a virtual space on the display (420) based on the execution of the XR application (442). The XR plug-in (444-1) included in the XR application (442) may be referenced by the XR plug-in (444) of the XR system UI (441). Descriptions of the XR plug-in (444-1) that overlap with the description of the XR plug-in (444) may be omitted. The wearable device (200) may cause the execution of the virtual space manager (451) based on the execution of the XR application (442).

[0100] According to one embodiment, the wearable device (200) may provide a virtual space service based on the execution of the virtual space manager (451). For example, the virtual space manager (451) may include a platform (e.g., an Android platform) for supporting the virtual space service. The wearable device (200) may display the posture of a virtual object representing the user's posture rendered using data acquired through the sensor (430) based on the execution of the virtual space manager (451) on the display. The virtual space manager (451) may be referred to as a composition presentation manager (CPM).

[0101] For example, the virtual space manager (451) may include a runtime service (452). As an example, the runtime service (452) may be referred to as an OpenXR runtime module. The wearable device (200) may be used to provide at least one of a pose prediction function, a frame timing function, and / or a spatial input function to a user through the wearable device (200) based on the execution of the runtime service (452). As an example, the wearable device (200) may be used to perform rendering for a virtual space service to a user based on the execution of the runtime service (452). For example, an application (e.g., unity or an OpenXR native application) may be implemented based on the execution of the runtime service (452).

[0102] For example, the virtual space manager (451) may include a pass-through manager (453). Based on the execution of the pass-through manager (453), the wearable device (200) may display a screen representing a virtual space on the display (420), while another screen representing an actual space acquired through the camera (425) may be superimposed on at least a portion of the screen.

[0103] For example, the virtual space manager (451) may include an input manager (454). Based on the execution of the input manager (454), the wearable device (200) may identify data (e.g., sensor data) acquired by executing one or more programs included in the recognition service layer (470). The wearable device (200) may initiate execution of at least one of the functions of the wearable device (200) using the acquired data.

[0104] For example, the perception abstract layer (460) can be used for data exchange between the virtual space manager (451) and the perception service layer (470). From the perspective of being used for data exchange between the virtual space manager (451) and the perception service layer (470), the perception abstract layer (460) can be referred to as an interface. For example, the perception abstract layer (460) can be referenced as OpenPX. The perception abstract layer (460) can be used for a perception client and a perception service.

[0105] According to one embodiment, the recognition service layer (470) may include one or more programs for processing data acquired from a sensor (430) (or a camera (425)). The one or more programs may include at least one of a position tracker (471), a space recognizer (472), a gesture tracker (473), an eye tracker (474), and / or a face tracker (475). The type and / or number of the one or more programs included in the recognition service layer (470) are not limited to those illustrated in FIG. 4.

[0106] For example, the wearable device (200) can identify the pose of the wearable device (200) using the sensor (430) based on the execution of the position tracker (471). The wearable device (200) can identify the 6 degrees of freedom pose (6 dof pose) of the wearable device (200) using data acquired using the camera (425) and the IMU based on the execution of the position tracker (471). The position tracker (471) can be referred to as a head tracking (HeT) module.

[0107] For example, the wearable device (200) may be used to construct a three-dimensional virtual space around the wearable device (200) (or a user of the wearable device (200)) based on the execution of the space recognizer (472). The wearable device (200) may reconstruct the three-dimensional surroundings of the wearable device (200) using data acquired using the camera (425) based on the execution of the space recognizer (472). The wearable device (200) may identify at least one of a plane, a slope, and stairs based on the three-dimensionally reconstructed surroundings of the wearable device (200) based on the execution of the space recognizer (472). The space recognizer (472) may be referred to as a scene understanding (SU) module.

[0108] For example, the wearable device (200) may be used to identify (or recognize) a pose and / or gesture of a hand of a user of the wearable device (200) based on the execution of the gesture tracker (473). As an example, the wearable device (200) may identify a pose and / or gesture of a hand of a user using data acquired from a sensor (430) based on the execution of the gesture tracker (473). As an example, the wearable device (200) may identify a pose and / or gesture of a hand of a user based on data (or images) acquired using a camera based on the execution of the gesture tracker (473). The gesture tracker (473) may be referred to as a hand tracking (HaT) module and / or a gesture tracking module.

[0109] For example, the wearable device (200) may identify (or track) eye movements of a user of the wearable device (200) based on the execution of the gaze tracker (474). As an example, the wearable device (200) may identify eye movements of the user using data acquired from at least one sensor based on the execution of the gaze tracker (474). As an example, the wearable device (200) may identify eye movements of the user based on data acquired using a camera (e.g., the gaze tracking camera (260-1) of FIGS. 2A and 2B) and / or an infrared light emitting diode (IR LED) based on the execution of the gaze tracker (474). The gaze tracker (474) may be referred to as an eye tracking (ET) module and / or a gaze tracking module.

[0110] For example, the recognition service layer (470) of the wearable device (200) may further include a face tracker (475) for tracking the user's face. For example, the wearable device (200) may identify (or track) the movement of the user's face and / or the user's expression based on the execution of the face tracker (475). The wearable device (200) may estimate the user's expression based on the movement of the user's face based on the execution of the face tracker (475). As an example, the wearable device (200) may identify the movement of the user's face and / or the user's expression based on data (e.g., an image) acquired using a camera based on the execution of the face tracker (475).

[0111] For the embodiments described below, the wearable device (200) of FIG. 4 may be referenced. For example, the embodiments described below may be performed by the processor (410) of the wearable device (200) of FIG. 4.

[0112] FIG. 5A illustrates an example of a virtual space displayed through a display according to one embodiment.

[0113] Referring to FIG. 5A, the wearable device (200) may include a camera (425) positioned toward the front of the user (510) when the device is worn by the user (510). The front of the user (510) may include the direction in which the user's (510) head and / or the user's (510) gaze is directed. According to one embodiment, the wearable device (200) may include a sensor (430) for identifying the motion of the user's (510) head and / or the wearable device (200) when the device is worn by the user (510).

[0114] The processor (410) of the wearable device (200) can identify the angle of the wearable device (200) based on data from the sensor (430). In order to provide a user interface (UI) based on virtual reality (VR), augmented reality (AR), and / or mixed reality (MR) to a user (510) wearing the wearable device (200), the processor (410) can control the camera (425) and / or the sensor (430). The UI can be related to the wearable device (200), a metaverse service provided by a server connected to the wearable device (200), and / or a notification service.

[0115] In one embodiment, the processor (410) may execute functions related to augmented reality (AR) and / or mixed reality (MR). When the user (510) wears the wearable device (200), the processor (410) may include at least one lens positioned adjacent to the eyes of the user (510). Ambient light passing through the lens may be combined (or mixed) with light emitted from the display (420) of the wearable device (200). A display area of ​​the display (420) may be formed within the lens through which the ambient light passes. Since the processor (410) combines the ambient light and the light emitted from the display (420), the user (510) may see an image that is a mixture of a real object recognized by the ambient light and a virtual object formed by the light emitted from the display (420).

[0116] According to one embodiment, a wearable device (200) may perform functions related to video see-through (VST) and / or virtual reality (VR). When a user (510) wears the wearable device (200), the wearable device (200) may include a housing that covers the eyes of the user (510). In the state, the wearable device (200) may include a display (420) disposed on a first surface facing the eyes (e.g., the first surface (310) of FIG. 3A). The wearable device (200) may include a camera (425) (or cameras (260-7, 260-8, 260-9, 260-10, 260-11, 260-12) of FIG. 3B) disposed on a second surface opposite to the first surface (e.g., the second surface (320) of FIG. 3A). Using the camera (425), the processor (410) may acquire frame images including ambient light. The processor (410) may output the frame images to a display (420) disposed on the first surface, thereby allowing a user (510) to recognize the ambient light through the display (420). A display area of ​​the display (420) disposed on the first surface may be formed by one or more pixels included in the display (420). The processor (410) can synthesize a virtual object within frame images output through the display (420) to enable the user (510) to recognize the virtual object together with a real object recognized by ambient light.

[0117] According to one embodiment, the wearable device (200) can provide a user experience based on mixed reality (MR) by utilizing a virtual space. The processor (410) of the wearable device (200) can recognize an external space (e.g., a real space) in which the wearable device (200) is included, and generate a virtual space mapped to the external space. The spatial recognition performed by the processor (410) can include simultaneous localization and mapping (SLAM) and / or spatial mapping (e.g., scene understanding).

[0118] According to one embodiment, the processor (410) may display (or provide) a virtual space (520) through the display (420). For example, the virtual space (520) may be configured to represent at least one virtual space. For example, the virtual space (520) may be referred to as a home space. A user (510) of the processor (410) may identify visual objects corresponding to the virtual spaces displayed within the virtual space (520).

[0119] For example, the processor (410) may display the virtual space (520) through the display (420) based on identifying an input from the user (510) regarding the start of a service regarding the virtual space. The user (510) (or an avatar corresponding to the user) may execute at least one of a plurality of applications that can be provided through the wearable device (200) within the virtual space (520).

[0120] For example, the processor (410) may provide at least one virtual space. For example, the processor (410) may provide not only the virtual space (520), but also a first virtual space, a second virtual space, a third virtual space, and a fourth virtual space. The visual object (521) may correspond to the first virtual space. The visual object (522) may correspond to the second virtual space. The visual object (523) may correspond to the third virtual space. The visual object (524) may correspond to the fourth virtual space.

[0121] According to one embodiment, the processor (410) may switch the virtual space provided to the user (510) based on an input for a visual object corresponding to the virtual space. For example, the processor (410) may switch the virtual space provided to the user (510) from the virtual space (520) to the first virtual space based on an input for a visual object (521) corresponding to the first virtual space. For example, the processor (410) may provide the first virtual space through the display (420) based on an input for a visual object (521) corresponding to the first virtual space.

[0122] According to one embodiment, the processor (410) can display (or provide) an object (530) within a virtual space (520). The processor (410) can display the object (530) on a designated area within the virtual space (520). The processor (410) can display the object (530) on the designated area independently of the motion of the user (510). For example, the designated area may not be a fixed area within the virtual space (520). The location of the designated area may change within the virtual space (520) depending on the user's gaze. The processor (410) can display the object (530) at the same location within a portion of the virtual space (520) identified by the user (510) (or a screen displayed through the display (420)) even when the user's gaze moves. According to an embodiment, the processor (410) may maintain the display of the object (530) even when the virtual space provided (or displayed) to the user (510) changes from the virtual space (520).

[0123] According to one embodiment, the processor (410) may conditionally display an object (530) based on the gaze of the user (510). The processor (410) may display the object (530) in response to a specified utterance of the user (510) and / or a gesture of pressing a button formed on the wearable device (200).

[0124] According to one embodiment, the object (530) may be referenced as one of a menu, a universal menu, a task bar, and a state bar.

[0125] For example, the processor (410) may display one or more elements (or icons) indicating the status of the wearable device (200) within the object (530). For example, the processor (410) may display an element indicating the state of charge (SOC) of the battery included in the wearable device (200) within the object (530). The processor (410) may display an element indicating the type of network (e.g., Wi-Fi, LTE, 5G and / or 6G) to which the wearable device (200) is connected within the object (530). The processor (410) may display an element and / or text indicating the current time within the object (530). The processor (410) may display an element (534) for indicating account information of a user (510) logged into the wearable device (200) within the object (530).

[0126] According to one embodiment, the processor (410) may display options for controlling the virtual space provided through the display area within the object (530).

[0127] For example, the processor (410) may display an element (531) for displaying a list of applications installed on the wearable device (200) in the object (530). In response to detecting a gaze and / or finger of the user (510) directed toward the element (531) for a specified period of time, the processor (410) may identify an input indicating selection of the element (531). In response to the input, the processor (410) may display a list of applications corresponding to the element (531).

[0128] For example, the processor (410) may display an element (532) for displaying a list of one or more virtual spaces provided (or generated) by the wearable device (200) in the object (530). The processor (410) may display the list of one or more virtual spaces (e.g., visual objects (521) to (524)) based on detecting a gesture for selecting the element (532) (e.g., a gaze and / or a finger of the user (510) directed toward the element (532).

[0129] For example, the processor (410) may display an element (533) for displaying a list of applications running within a virtual space provided through a display area in the object (530).

[0130] FIG. 5b illustrates an example of a portion of a virtual space displayed according to the direction of a user's gaze, according to one embodiment.

[0131] Referring to FIG. 5B, the processor (410) can identify the user's motion using the sensor (430). For example, the processor (410) can identify the motion of a part of the user's body (e.g., the head). The processor (410) can identify the 6 degrees of freedom pose (6 dof pose) of the wearable device (200) using the sensor (430).

[0132] For example, the processor (410) can identify the direction of the user's gaze based on the rotational direction of the wearable device (200). The processor (410) can identify the rotation of the wearable device (200) based on one of the x-axis, the y-axis, and the z-axis.

[0133] For example, the processor (410) can identify the rotation of the wearable device (200) around the x-axis. The processor (410) can identify the pitching (or nodding) of the user's head based on the rotation of the wearable device (200) identified around the x-axis.

[0134] For example, the processor (410) can identify the rotation of the wearable device (200) about the y-axis. The processor (410) can identify the yawing (or horizontal shaking) of the user's head based on the rotation of the wearable device (200) identified about the y-axis.

[0135] For example, the processor (410) can identify the rotation of the wearable device (200) around the z-axis. The processor (410) can identify the rolling (or tilting) of the user's head based on the rotation of the wearable device (200) identified around the z-axis.

[0136] According to one embodiment, since the virtual space (580) is configured in three dimensions, the processor (410) may not be able to display the entire virtual space (580) through the display (420). Therefore, the processor (410) may display a portion of the virtual space (580) based on the direction of the user's gaze.

[0137] For example, the processor (410) can identify the direction of the user's gaze within the virtual space (580). In FIG. 5b, for convenience of explanation, an example is shown in which the direction of the user's gaze changes based on the y-axis, but the present invention is not limited thereto.

[0138] For example, the processor (410) can identify that the direction of the user's gaze within the virtual space (580) is the direction (551). Based on the direction (551), the processor (410) can display a first portion (561) of the virtual space (580) through the display (420).

[0139] For example, the processor (410) can identify that the direction of the user's gaze within the virtual space (580) is the direction (552). Based on the direction (552), the processor (410) can display a second portion (562) of the virtual space (580) through the display (420).

[0140] For example, the processor (410) can identify that the direction of the user's gaze within the virtual space (580) is direction (553). Based on the direction (553), the processor (410) can display a third part (563) of the virtual space (580) through the display (420).

[0141] For example, the processor (410) can identify that the direction of the user's gaze within the virtual space (580) is the direction (554). Based on the direction (554), the processor (410) can display a fourth portion (564) of the virtual space (580) through the display (420).

[0142] In the above-described embodiment, an example is shown in which a portion of the virtual space (580) is displayed based on four directions (551, 552, 553, 554). However, this is merely exemplary, and the direction of the user's gaze may be configured continuously. The processor (410) may continuously change a portion of the virtual space (580) displayed through the display (420) based on the continuously changing direction of the user's gaze.

[0143] FIG. 6 illustrates an example of a virtual space for executing at least one application, according to one embodiment.

[0144] Referring to FIG. 6, the processor (410) may provide a virtual space (600) to a user (510). The virtual space (600) may include a visual object (610) for at least one interface for at least one application. The visual object (610) may be used to provide at least one interface for at least one application. For example, the visual object (610) may be referred to as a dash (or dashboard).

[0145] The processor (410) may display an interface relating to an application within the visual object (610). For example, the processor (410) may display a first interface (601), a second interface (602), and a third interface (603) on the visual object (610). The first interface (601) may be associated with a first application. The second interface (602) may be associated with a second application. The third interface (603) may be associated with a third application.

[0146] The processor (410) can execute one or more applications simultaneously within the virtual space (600). The processor (410) can display one or more interfaces related to one or more applications being executed within a visual object (610) included in the virtual space (600). A user (510) of the wearable device (200) can simultaneously identify (or use) one or more applications within the visual object (610).

[0147] According to one embodiment, the processor (410) may display an object (530) within the virtual space (600). For example, the object (530) may correspond to the object (530) illustrated in FIG. 5.

[0148] According to one embodiment, the virtual space (600) may be related to the work environment of the user (510). The processor (410) may provide at least one of an infinite screen, a multi-window, and / or a 3D object within the virtual space (600), thereby providing a highly productive experience to the user (510).

[0149] According to an embodiment, the processor (410) may display an external electronic device connected to the wearable device (200) in the virtual space (600). For example, the processor (410) may display the external electronic device based on video see through (VST) and / or optical see through (OST). By displaying the external electronic device in the virtual space (600), the processor (410) may provide a multi-device experience (MDE) to the user (510).

[0150] In some embodiments, the processor (410) may provide the user (510) with the ability to change at least one of an object, lighting, theme, and / or environment within the virtual space (600). The processor (410) may provide the user (510) with an option to personalize the virtual space (600).

[0151] FIG. 7 illustrates an example of a virtual space for providing an interface for an application, according to one embodiment.

[0152] Referring to FIG. 7, the processor (410) can provide a virtual space (700) to a user (510). For example, the virtual space (700) can be configured to provide an interface (710) for a specified application.

[0153] For example, the processor (410) may provide a virtual space (700) based on the execution of a specified application. For example, the virtual space (700) may be associated with one application.

[0154] In some embodiments, the use of other applications within the virtual space (700) may be restricted. The processor (410) may display an element for switching the virtual space for use by other applications via an object (530). Based on an input to the element, the processor (410) may switch the virtual space (700) to another virtual space (e.g., the virtual space (600) of FIG. 6) and display an interface for the other application.

[0155] According to one embodiment, while the processor (410) provides an interface (710) for one application through the virtual space (700), it may not be able to provide information about another application. The user (510) may not be able to check information about another application within the virtual space (700) without switching the virtual space (700) to another virtual space. Therefore, in the following specification, an embodiment for displaying information (e.g., interface or execution information) about an application running in another virtual space (e.g., virtual space (600) of FIG. 6) within the virtual space (700) will be described.

[0156] FIG. 8a illustrates an example of a virtual space being converted, according to one embodiment.

[0157] FIG. 8b illustrates an example of a virtual space being switched, according to one embodiment.

[0158] Referring to FIGS. 8A and 8B, the processor (410) of the wearable device (200) can provide a plurality of virtual spaces to the user (510). For example, the processor (410) can provide virtual spaces (820) to (825) to the user (510).

[0159] The processor (410) may provide a function for moving through multiple virtual spaces. For example, the processor (410) may display a virtual space (820) (or a portion of the virtual space (820)), which is an initial screen, through the display (420). The virtual space (820) may correspond to the virtual space (520) illustrated in FIG. 5A.

[0160] For example, each of virtual space (821) and virtual space (822) may be configured to provide an interface for a single application. Each of virtual space (821) and virtual space (822) may correspond to virtual space (700) of FIG. 7. For example, each of virtual space (823), virtual space (824), and virtual space (825) may correspond to virtual space (600) of FIG. 6.

[0161] Referring to FIG. 8A, the processor (410) may provide a path for providing one of other virtual spaces within the virtual space (820). For example, the processor (410) may display a visual object corresponding to each virtual space within the virtual space (820). Based on identifying an input for a visual object corresponding to the virtual space (821), the processor (410) may switch the virtual space displayed through the display (420) from the virtual space (820) to the virtual space (821). For example, the processor (410) may receive an input from the user (510) for switching the virtual space displayed through the display (420) from the virtual space (821) to the virtual space (820) in order to switch the virtual space displayed through the display (420) from the virtual space (821) to the virtual space (823). After the virtual space displayed through the display (420) is converted to the virtual space (820), the processor (410) may receive an input for converting the virtual space displayed through the display (420) from the virtual space (820) to the virtual space (823). Based on the input for converting the virtual space displayed through the display (420) from the virtual space (820) to the virtual space (823), the processor (410) may display the virtual space (823) through the display (420). Therefore, the processor (410) may always need to move to the virtual space (820) in order to convert the virtual space.

[0162] Referring to FIG. 8B, unlike FIG. 8A, the processor (410) may provide a function for switching virtual spaces without using the virtual space (820). For example, the processor (410) may display the virtual space (823) without displaying the virtual space (820) based on identifying an input for switching the virtual space displayed through the display (420) from the virtual space (821) to the virtual space (823).

[0163] Although not illustrated, the processor (410) can of course move from the virtual space (821) to the virtual space (825) based on identifying an input for switching the virtual space displayed through the display (420) from the virtual space (821) to the virtual space (825).

[0164] According to the following embodiment, the processor (410) can switch virtual spaces without going through a virtual space (820) corresponding to the home virtual space, as shown in FIG. 8b.

[0165] Figure 9 illustrates a flowchart of the operation of a wearable device according to one embodiment. In the following embodiments, the operations may be performed sequentially, but are not necessarily sequential. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0166] Referring to FIG. 9, in operation 910, the processor (410) may identify at least one application running within the first virtual space while the first virtual space is displayed. For example, the first virtual space may be configured to run at least one application. For example, the first virtual space may correspond to the virtual space (600) illustrated in FIG. 6. The processor (410) may run at least one application within the first virtual space. For example, the processor (410) may display at least one interface related to at least one application running within the first virtual space. For example, the first virtual space may be configured to include at least one area for running at least one application. The processor (410) may display a visual object (e.g., the visual object (610) illustrated in FIG. 6) including at least one area within the first virtual space.

[0167] At operation 920, the processor (410) may identify an input for executing a first application within a first virtual space. For example, the input for executing the first application may be an input for switching a virtual space displayed through the display (420) from the first virtual space to a second virtual space provided according to the first application. For example, the first application may be distinguished from at least one application. For example, the input may be referred to as an input for switching from the first virtual space to a second virtual space related to the first application.

[0168] For example, the input may be configured in various ways. The input may be performed through a part of the user's body, an external device (e.g., a remote controller, another wearable device, a keyboard, a mouse), and / or voice input.

[0169] In operation 930, the processor (410) may switch the first virtual space to a second virtual space provided according to the first application. For example, based on the execution of the first application, the processor (410) may switch the first virtual space to a second virtual space provided according to the first application. For example, based on an input for executing the first application, the processor (410) may execute the first application. Based on the execution of the first application, the processor (410) may display the second virtual space provided through the first application.

[0170] For example, the second virtual space may be associated with the first application. The second virtual space may be configured to provide an interface for the first application. For example, the first application may provide an interface configured in 360 degrees of omnidirectional view. For example, the first application may provide an interface configured based on a specified viewing angle. For example, the second virtual space may correspond to the virtual space (700) illustrated in FIG. 7.

[0171] In operation 940, the processor (410) may identify a second application running within the first virtual space among at least one application while the second virtual space is displayed.

[0172] In one embodiment, a second application running while a second virtual space is displayed may be identified among at least one application. For example, the processor (410) may identify a second application running while a second virtual space is displayed among at least one application running within the first virtual space.

[0173] For example, the processor (410) may identify an application running in the background among at least one application running within the first virtual space. A second application among the at least one application may continue to operate in the background even when the virtual space is switched. For example, some of the at least one application may be terminated when the virtual space is switched.

[0174] In operation 950, the processor (410) may identify the motion of the wearable device (200) while displaying the first portion of the second virtual space. For example, the processor (410) may identify the motion of the wearable device (200) while displaying the first portion of the second virtual space based on the first direction.

[0175] According to one embodiment, the processor (410) may display a first portion of a second virtual space based on a first direction. For example, the first direction may be referred to as a frontal direction. The processor (410) may display a first portion of the second virtual space based on identifying an input for transitioning from the first virtual space to the second virtual space.

[0176] For example, the processor (410) may not be able to display the entire second virtual space through the display (420). Therefore, the processor (410) may identify that the direction in which the user's gaze is directed is the first direction. The processor (410) may display the first portion of the second virtual space based on the first direction.

[0177] According to one embodiment, the processor (410) can identify the motion of the wearable device (200). The processor (410) can identify the motion of the wearable device (200) using a sensor (430) (e.g., a gyro sensor, an inertial sensor). The motion of the wearable device (200) can be related to the motion of the wearable device (200) described in FIG. 5B.

[0178] For example, the processor (410) can identify the rotation of the wearable device (200) with respect to the y-axis illustrated in FIG. 5B. The processor (410) can identify the yawing of the wearable device (200) (or the user's head).

[0179] In operation 960, the processor (410) may display a visual object related to the second application within a second portion of the second virtual space. For example, the processor (410) may display a visual object related to the second application within the second portion of the second virtual space based on a second direction changed from a first direction according to the motion of the wearable device (200).

[0180] According to one embodiment, the processor (410) can identify a second direction changed from a first direction according to the motion of the wearable device (200). The processor (410) can identify that the user's field of view (FoV) has changed from the first FoV to the second FoV. The processor (410) can display a first portion of the second virtual space based on the first FoV. The processor (410) can display a second portion of the second virtual space based on the second FoV.

[0181] The processor (410) may display a visual object related to the second application within the second portion of the second virtual space. For example, the visual object related to the second application may include one of an interface related to the second application, a visual object representing a function of the second application, or a visual object for moving to the first virtual space where the second application is executed.

[0182] In one embodiment, a visual object relating to a second application may include an interface relating to the second application. The processor (410) may display an interface relating to the second application running in the first virtual space within a second portion of the second virtual space. The processor (410) may provide continuity between virtual spaces based on the user (510) displaying an interface relating to the second application running in the first virtual space within the second virtual space.

[0183] For example, the interface for the second application may be configured to be identical or similar to the interface displayed within the first virtual space. For example, the interface for the second application may be configured as a reduced interface. For example, the processor (410) may display the interface for the second application in the form of a pop-up window. For example, the interface for the second application may be displayed transparently within the second virtual space for the first application. The processor (410) may display the interface for the second application in a different layer from the second virtual space. The processor (410) may display the interface for the second application within the second virtual space without restricting the view of the second virtual space.

[0184] According to one embodiment, a visual object related to a second application may include a visual object representing a function of the second application. For example, the second application may be configured to perform an alarm function. The processor (410) may display a visual object in the shape of a clock representing the alarm function within the second virtual space. For example, the second application may be configured to perform a music playback function. The processor (410) may display a visual object in the shape of a musical note (or play button) representing music playback within the second virtual space.

[0185] For example, the processor (410) may display an interface for a second application running within the first virtual space, within the second virtual space, based on an input for a visual object.

[0186] In one embodiment, the visual object related to the second application may include a visual object for moving to a first virtual space in which the second application is executed. Based on an input for the visual object related to the second application, the processor (410) may switch the virtual space displayed through the display (420) from the first virtual space to the second virtual space.

[0187] FIG. 10A illustrates an example of operation of a wearable device for displaying a visual object related to a second application within a second virtual space, according to one embodiment.

[0188] Referring to FIG. 10A, in state (1001), the processor (410) may display a first virtual space (1010) (or a portion of the first virtual space (1010). The processor (410) may identify at least one application running within the first virtual space (1010). The processor (410) may display at least one interface related to the at least one application. The processor (410) may display interface (1011), interface (1012), and interface (1013). According to an embodiment, the processor (410) may display an object (530) within the first virtual space (1010).

[0189] According to one embodiment, the processor (410) can identify an input for transitioning from a first virtual space (1010) to a second virtual space (1020) related to a first application. Based on identifying the input, the processor (410) can identify a second application to be executed while the second virtual space is displayed. For example, the object (530) can include an element for transitioning to the second virtual space (1020) (or an element for executing the first application). Based on identifying the input for the element, the processor (410) can identify the second application to be executed while the second virtual space (1020) is displayed.

[0190] For example, the second application may be an application corresponding to the interface (1012). For example, the second application may be executed while the second virtual space (1020) is displayed. The second application may be executed in the background while the second virtual space (1020) is displayed. For example, among at least one application executed within the first virtual space (1010), the remaining applications, excluding the second application, may not be executed while the second virtual space (1020) is displayed.

[0191] For example, an object (530) within a first virtual space (1010) may include an element corresponding to a first application. The processor (410) may identify an input for transitioning from the first virtual space (1010) to a second virtual space (1020) related to the first application based on identifying an input for the element corresponding to the first application. The input may be performed through a part of the user's body, an external device (e.g., a remote controller, another wearable device, a keyboard, a mouse), and / or a voice input.

[0192] According to one embodiment, the processor (410) may change the state of the wearable device (200) from state (1001) to state (1002) based on identifying an input for transitioning from a first virtual space (1010) to a second virtual space (1020).

[0193] In state (1002), the processor (410) can display a first part (1021) of a second virtual space (1020) based on a first direction (1031). The processor (410) can identify that the gaze of the user (510) of the wearable device (200) is directed toward the first direction (1031). The processor (410) can identify a first part (1021) of the second virtual space (1020) based on the first direction (1031). The processor (410) can display the identified first part (1021) of the second virtual space (1020) through the display (420).

[0194] For example, the processor (410) can display an object (530) within a first portion (1021) of a second virtual space (1020). The processor (410) can display the object (530) displayed in the first virtual space (1010) in the second virtual space (1020) in the state (1001).

[0195] According to one embodiment, the processor (410) can identify the motion of the wearable device (200). The processor (410) can identify a second direction (1032) changed from a first direction (1031) according to the motion of the wearable device (200). The processor (410) can identify that the direction of the user's gaze has changed from the first direction (1031) to the second direction (1032) based on the motion of the wearable device (200). The processor (410) can change the state of the wearable device (200) from state (1002) to state (1003) based on the motion of the wearable device (200).

[0196] In state (1003), the processor (410) can display a visual object (1025) related to a second application within a second portion (1022) of a second virtual space (1020) based on a second direction (1032). In state (1002), the processor (410) can also display an object (530) displayed in a first portion (1021) of the second virtual space (1020) within the second portion (1022) of the second virtual space (1020).

[0197] For example, the second application may include an application for playing music. The processor (410) may display a visual object (1025) based on the function of the second application. In FIG. 10A, an example in which the visual object (1025) is displayed when the second application is an application for playing music is illustrated, but the present invention is not limited thereto. The second application may be one of an application for an alarm, an application for managing schedules, and an application for working on documents. For example, when the second application is an application for an alarm, the shape of the visual object (1025) may be set to the shape of an alarm clock. For example, when the second application is an application for managing schedules, the shape of the visual object (1025) may be set to the shape of a calendar. For example, when the second application is an application for working on documents, the shape of the visual object (1025) may be set to the shape of a document.

[0198] In one embodiment, the processor (410) can identify an input to a visual object (1025). Based on identifying the input to the visual object (1025), the processor (410) can display an interface (1012) for a second application within a designated area within a second portion (1022) of a second virtual space (1020). The processor (410) can display an interface (1012) for a second application running within the first virtual space (1010) within the second virtual space (1020) (or the second portion (1022) of the second virtual space (1020)). The processor (410) may provide a function to the user (510) to control the second application within the second virtual space (1020) by displaying an interface (1012) regarding the second application running within the first virtual space (1010) in the second virtual space (1020) (or a second portion (1022) of the second virtual space (1020)).

[0199] According to one embodiment, the processor (410) can identify an input for a visual object (1025). Based on identifying the input for the visual object (1025), the processor (410) can perform a transition from a second virtual space (1020) to a first virtual space (1010) in which a second application is executed. Based on identifying the input for the visual object (1025), the processor (410) can change a virtual space displayed through the display (420) from the second virtual space (1020) to the first virtual space (1010). For example, based on identifying the input for the visual object (1025), the processor (410) can change a state of the wearable device (200) from state (1003) to state (1001). The processor (410) may display an interface (1012) for a second application within the first virtual space (1010). Depending on the embodiment, the processor (410) may display not only the interface (1012), but also the interface (1011) and the interface (1013) within the first virtual space (1010).

[0200] According to one embodiment, the visual object (1025) may be positioned at a fixed position within the second virtual space (1020). The processor (410) may identify that an avatar corresponding to the user (510) is positioned on the visual object (1025). Based on identifying that the avatar corresponding to the user (510) is positioned on the visual object (1025), the processor (410) may switch the virtual space displayed through the display (420) from the second virtual space (1020) to the first virtual space (1010). For example, based on identifying that the avatar corresponding to the user (510) is positioned on the visual object (1025), the processor (410) may provide a notification to the user (510) indicating whether to switch the virtual space. The processor (410) can switch the virtual space displayed through the display (420) from the second virtual space (1020) to the first virtual space (1010) based on identifying an input from the user (510) that instructs switching of the virtual space. The processor (410) can perform the space transition by switching the virtual space displayed through the display (420) from the second virtual space (1020) to the first virtual space (1010). The processor (410) can switch the virtual space displayed through the display (420) from the second virtual space (1020) to the first virtual space (1010) without going through the home virtual space corresponding to the virtual space (820) illustrated in FIG. 8A.

[0201] According to an embodiment, the processor (410) may display a visual object (not shown) including an interface (1011), an interface (1012), and an interface (1013) within a second portion (1022) of a second virtual space (1020). The processor (410) may display interfaces running within a first virtual space (1010) distinct from the second virtual space (1020) within a second portion (1022) of the second virtual space (1020).

[0202] FIG. 10b illustrates an example of operation of a wearable device to display a visual object related to a second application within a second virtual space, according to one embodiment.

[0203] Referring to FIG. 10B, in state (1006), the processor (410) may display a first virtual space (1060) (or a portion of the first virtual space (1060). The processor (410) may identify at least one application running within the first virtual space (1060). The processor (410) may display at least one interface related to the at least one application. The processor (410) may display interface (1061), interface (1062), and interface (1063). According to an embodiment, the processor (410) may display an object (530) within the virtual space (1060). For example, the location at which the object (530) is displayed may be set in various ways. For example, the object (530) may be displayed in at least one of an edge area or an upper area of ​​the virtual space (e.g., the first virtual space (1060)).

[0204] For example, the processor (410) may display the first virtual space (1060) (or a portion of the first virtual space (1060)) based on the direction (1068). The processor (410) may identify that the direction of the user's gaze is the direction (1068) and display the first virtual space (1060) based on the viewing angle (1069). Depending on the embodiment, the size of the viewing angle (1069) may be changed based on at least one of the type of the virtual space or the type of the application being executed.

[0205] According to one embodiment, the processor (410) can identify an input (1065) for transitioning from a virtual space (1060) to a second virtual space (1070) relating to a first application.

[0206] For example, input (1065) may include input to an interface (1061) for a first application. For example, input (1065) may be performed through an external electronic device (e.g., a controller) connected to the wearable device (200). For example, input (1065) may be performed through a part of the body of a user of the wearable device (200), an external device (e.g., a remote controller, another wearable device, a keyboard, a mouse), and / or voice input.

[0207] According to one embodiment, the processor (410) may identify a second application running while the second virtual space (1070) is displayed based on identifying an input (1065) to the interface (1061).

[0208] For example, the second application may be an application corresponding to the interface (1062). For example, the second application may be executed within the first virtual space (1060) while the second virtual space (1070) is displayed. The second application may be executed in the background while the second virtual space (1070) is displayed. For example, among at least one application executed within the first virtual space (1060), the remaining applications, excluding the second application, may not be executed while the second virtual space (1070) is displayed.

[0209] According to one embodiment, the processor (410) may change the state of the wearable device (200) from state (1006) to state (1007) based on identifying the input (1065).

[0210] In state (1007), the processor (410) can display a first portion (1075) of a second virtual space (1070) based on a direction (1068). The processor (410) can identify that the gaze of the user (510) of the wearable device (200) is directed toward the direction (1068). The processor (410) can identify a first portion (1075) of the second virtual space (1070) based on the direction (1068). The processor (410) can display the identified first portion (1075) of the second virtual space (1070) through the display (420). For example, the processor (410) can change the viewing angle for displaying the first portion (1075) of the second virtual space (1070) based on identifying an input (1065) to the interface (1061). The processor (410) can set the viewing angle for displaying the first portion (1075) of the second virtual space (1070) to the viewing angle (1079). As an example, the processor (410) can change the viewing angle to be wider based on the execution of the first application. The processor (410) can provide a wider viewing angle (e.g., viewing angle (1079)) for displaying the second virtual space (1070) that has been switched from the first virtual space (1060) for displaying at least one application.

[0211] For example, the processor (410) can identify a second application running in the first virtual space (1060) within the first portion (1075) of the second virtual space (1070). The processor (410) can display an object (1071) related to the second application running in the first virtual space (1060) within the first portion (1075) of the second virtual space (1070). A user of the wearable device (200) can control the second application running in the first virtual space (1060) using the object (1071).

[0212] For example, the processor (410) can remove an object (530) displayed in the virtual space (1060) from the first portion (1075) of the second virtual space (1070). The processor (410) can display a reduced version of the object (1072). Although not illustrated, the processor (410) can display the object (530) within the first portion (1075) of the second virtual space (1070) based on an input for the object (1072). Although not illustrated, the processor (410) can display the object (530) within the first portion (1075) of the second virtual space (1070) based on identifying that the user's gaze is positioned on the object (1072).

[0213] According to one embodiment, the processor (410) can identify the motion of the wearable device (200). The processor (410) can identify a direction (1085) changed from direction (1068) based on the motion of the wearable device (200). The processor (410) can identify that the direction of the user's gaze has changed from direction (1068) to direction (1085) based on the motion of the wearable device (200). The processor (410) can change the state of the wearable device (200) from state (1007) to state (1008) based on the motion of the wearable device (200). According to an embodiment, the processor (410) can change the state of the wearable device (200) from state (1007) to state (1008) based on the direction of the user's gaze being rotated beyond a threshold angle.

[0214] In state (1008), the processor (410) can display a second portion (1076) of the second virtual space (1070). The processor (410) can display the second portion (1076) of the second virtual space (1070) while the screen is switched from the first portion (1075) of the second virtual space (1070) to the third portion (1077) of the second virtual space (1070).

[0215] In state (1008), the second portion (1076) of the second virtual space (1070) may include a portion of the first portion (1075) and a portion of the third portion (1077). For example, the processor (410) may highlight an area (1082) representing an outer perimeter of the first portion (1075). For example, the processor (410) may highlight the area (1082) by changing the color of the area (1082) based on a gradation technique.

[0216] The processor (410) can display an object (1081) related to a second application running in the first virtual space (1060) within the area (1082). The processor (410) can remove the object (1071) and display the object (1081) within the area (1082) while the screen transitions from the first part (1075) of the second virtual space (1070) to the third part (1077) of the second virtual space (1070). For example, the processor (410) can maintain the display of the object (1072) while the screen transitions from the first part (1075) of the second virtual space (1070) to the third part (1077) of the second virtual space (1070).

[0217] According to one embodiment, the processor (410) can identify a direction (1095) changed from direction (1085) based on the motion of the wearable device (200). The processor (410) can identify that the direction of the user's gaze has changed from direction (1085) to direction (1095). Based on identifying that the direction of the user's gaze has changed from direction (1085) to direction (1095), the processor (410) can change the state of the wearable device (200) from state (1008) to state (1009).

[0218] In state (1009), the processor (410) can display a third portion (1077) of the second virtual space (1070). The processor (410) can display at least one interface displayed in the first virtual space (1060) within the third portion (1077) of the second virtual space (1070). For example, the processor (410) can display an interface (1091) corresponding to interface (1062) within the third portion (1077) of the second virtual space (1070). The processor (410) can display an interface (1092) corresponding to interface (1063) within the third portion (1077) of the second virtual space (1070). For example, the processor (410) may display the remaining interfaces except for the interface (1061) corresponding to the currently running first application among the interfaces (1061), (1062), and (1063) displayed in the state (1006). Accordingly, the processor (410) may display the interface (1091) corresponding to the interface (1062) and the interface (1092) corresponding to the interface (1063) within the third portion (1077) of the second virtual space (1070).

[0219] For example, the processor (410) may display at least one interface (e.g., interface (1091), interface (1092)) relating to at least one application running (or being run) in the first virtual space (1060) within a third portion (1077) of the second virtual space (1070) that is distinct from a first portion (1075) of the second virtual space (1070) for displaying functions relating to the first application.

[0220] According to an embodiment, the processor (410) may display an interface (e.g., interface (1091)) regarding a second application running within the first virtual space (1060) within the third portion (1077) of the second virtual space (1070).

[0221] According to one embodiment, the processor (410) may highlight an area (1093) representing an outer edge of a third portion (1077) to indicate that a screen related to a first application is being displayed within a first portion (1075) of a second virtual space (1070). For example, the processor (410) may highlight an area (1093) representing an outer edge of a third portion (1077) by changing the color of the area (1093). For example, the processor (410) may change the color of the area (1093) based on a gradation technique.

[0222] According to one embodiment, the processor (410) can display an object (530) within a third portion (1077) of a second virtual space (1070). The processor (410) can remove the object (1072) and display the object (530).

[0223] FIG. 11 illustrates an example of operation of a wearable device for displaying a visual object related to a second application within a second virtual space, according to one embodiment.

[0224] Referring to FIG. 11, in state (1101), the processor (410) may display a first virtual space (1110) through the display (420). For example, the processor (410) may identify at least one application running within the virtual space (1110). The processor (410) may display at least one interface (1111) related to at least one application. The state (1101) may correspond to the state (1001) of FIG. 10A.

[0225] According to one embodiment, the processor (410) can identify an input for transitioning from a virtual space (1110) to a second virtual space (1120) relating to a first application. Based on identifying the input, the processor (410) can identify a second application running while the second virtual space is displayed.

[0226] According to one embodiment, the processor (410) may change the state of the wearable device (200) from state (1101) to state (1102) based on identifying an input for transitioning from a first virtual space (1110) to a second virtual space (1120).

[0227] In state (1102), the processor (410) can display a first part (1121) of a second virtual space (1120) based on a first direction (1131). The processor (410) can identify that the gaze of the user (510) of the wearable device (200) is directed toward the first direction (1131). The processor (410) can identify a first part (1121) of the second virtual space (1120) based on the first direction (1131). The processor (410) can display the identified first part (1121) of the second virtual space (1120) through the display (420).

[0228] According to one embodiment, the processor (410) may display an object (530) within a first portion (1121) of a second virtual space (1120). The processor (410) may display an object (530) on a designated area within the first portion (1121) of the second virtual space (1120), the object (530) including one or more elements (e.g., element (1140), element (1150), element (1160), and element (1170)) relating to one or more applications running within one or more virtual spaces (e.g., first virtual space (1110)) distinct from the second virtual space (1120). For example, an element may be referenced as an affordance, an on-going affordance, or a visual object relating to a function of an application.

[0229] For example, the processor (410) may display an object (530) in a first virtual space (1110). Based on identifying that the virtual space displayed through the display (420) has changed from the first virtual space (1110) to the second virtual space (1120), the processor (410) may display one or more elements (e.g., element (1140), element (1150), element (1160), and element (1170)) relating to one or more applications running within one or more virtual spaces (e.g., the first virtual space (1110)) that are distinct from the second virtual space (1120) on the object (530).

[0230] For example, the processor (410) may identify an application running within a first virtual space (1110), which is one of one or more virtual spaces distinct from a second virtual space (1120). The processor (410) may display elements relating to the application running within the first virtual space (1110) within an object (530).

[0231] In one embodiment, each of the one or more elements may be displayed based on execution information of the application. For example, each of the one or more elements may be displayed based on at least one of the following: the type of the application, the function of the application, the progress of the function of the application, the operation time, and the time remaining until the completion of the operation.

[0232] For example, an application running within the first virtual space (1110) may be a music playback application. The processor (410) may display an element (1140) related to the music playback application running within the first virtual space (1110) within the object (530). As an example, the element (1140) may include an image (1141) representing the music playback application, an indicator (1142) corresponding to a timeline of the music being played, and an image (1143) indicating that music is being played. As an example, the indicator (1142) may be displayed as an overlay on the image (1141) and the image (1143).

[0233] For example, an application running within the first virtual space (1110) may be an alarm (or clock) application. The processor (410) may display an element (1150) related to the alarm application running within the first virtual space (1110) within the object (530). As an example, the element (1150) may include an image (1151) representing the alarm application, an indicator (1152) representing the ratio of the remaining time to the set time, and text (1153) representing the remaining time. As an example, the indicator (1152) may be displayed overlapping the image (1151) and the text (1153).

[0234] For example, an application running within the first virtual space (1110) may be a recording (or video recording) application. The processor (410) may display an element (1160) related to the recording (or video recording) application running within the first virtual space (1110) within the object (530). As an example, the element (1160) may include an image (1161) representing the recording (or video recording) application and text (1163) representing the time at which the recording (or video recording) was performed.

[0235] For example, an application running within the first virtual space (1110) may be an Internet application. The processor (410) may display an element (1170) related to the Internet application running within the first virtual space (1110) within the object (530). As an example, the element (1170) may include an image (1171) representing the Internet application, an indicator (1172) representing the progress of content being downloaded via the Internet, and text (1173) representing the remaining time. As an example, the indicator (1172) may be displayed as an overlay on the image (1171) and the text (1173).

[0236] As in the example described above, the processor (410) can identify one or more applications running within one or more virtual spaces distinct from the second virtual space (1120). For example, the processor (410) can identify one or more applications running in the background while the second virtual space (1120) is displayed. The processor (410) can display one or more elements related to the one or more applications within an object (530) displayed on a designated area within a first portion (1121) of the second virtual space (1120).

[0237] FIG. 12A illustrates an example of operation of a wearable device for displaying a visual object related to a second application within a second virtual space, according to one embodiment.

[0238] FIG. 12b illustrates an example of operation of a wearable device to display a visual object related to a second application within a second virtual space, according to one embodiment.

[0239] Referring to FIG. 12A, the processor (410) may display a second virtual space (1210). Within the second virtual space (1210), the processor (410) may display an interface (1211) related to a first application. According to an embodiment, the processor (410) may execute multiple applications within the second virtual space (1210). Although not illustrated, the processor (410) may display multiple interfaces related to multiple applications in the second virtual space (1210).

[0240] According to one embodiment, the processor (410) may display an object (530) within the second virtual space (1210). The object (530) may correspond to the object (530) illustrated in FIG. 5A. The processor (410) may identify a second application running in a first virtual space (not illustrated) that is distinct from the second virtual space (1210). For example, the processor (410) may run the second application in the background while displaying the second virtual space (1210). While displaying the second virtual space (1210), the processor (410) may display an element (531) related to the second application running within the object (530).

[0241] For example, the processor (410) may indicate that a second application within a virtual space other than the second virtual space (1210) is running in the background by displaying an element (531) within the object (530).

[0242] In one embodiment, the processor (410) can identify an input to an element (531). Based on the input to the element (531), the processor (410) can display an interface (1212) for a second application corresponding to the element (531).

[0243] For example, in response to an input to the element (531), the processor (410) may display an interface (1212) for a second application based on one of a prompt user interface (UI), a pop-up window, a split screen, and / or a picture in picture (PIP). As an example, the processor (410) may display an interface (1212) for a second application overlaid on an interface (1211) for a first application displayed within a second virtual space (1210).

[0244] For example, the processor (410) may, in response to an input for the element (531), distinguish a visual object (1213) (e.g., a visual object (610) of FIG. 6) for at least one interface within the second virtual space (1210). The processor (410) may divide an area of ​​the visual object (1213) for at least one interface within the second virtual space (1210) into a first area and a second area. The processor (410) may display an interface (1211) related to a first application running within the second virtual space (1210) within the first area. The processor (410) may display an interface (1212) related to a second application running within the first virtual space (not shown) within the second area.

[0245] Referring to FIG. 12B, the processor (410) can identify an input to an element (531) displayed within an object (530). For example, the processor (410) can identify an input to the element (531) via a remote controller (1251). For example, the processor (410) can identify an input to the element (531) via a part of the user's body (e.g., a hand or a finger).

[0246] For example, the processor (410) can communicate with the remote controller (1251). The processor (410) can identify the direction (1252) in which the remote controller (1251) is facing based on information received from the remote controller (1251). For example, the information received from the remote controller (1251) can include data on motion of the remote controller (1251) identified by an IMU included in the remote controller (1251). The motion of the remote controller (1251) can be generated by the hand of the user (510) holding the remote controller (1251).

[0247] For example, the processor (410) can identify an input to the element (531) based on the direction (1252) that the remote controller (1251) is facing. The processor (410) can display an object (e.g., a beam) that indicates the direction (1252) that the remote controller (1251) is facing. The processor (410) can identify an input to a button of the remote controller (1251) while the object points to the element (531). Based on the input to the button of the remote controller (1251), the processor (410) can display an interface (1212) for a second application in association with the element (531). For example, the interface (1212) can be displayed in an upper area of ​​an object (530) that includes the element (531). For example, the interface (1212) may be displayed within an area distinct from the visual object (1213).

[0248] According to an embodiment, the processor (410) may display an object (e.g., a beam) indicating a direction (1252) toward which the remote controller (1251) is facing. The processor (410) may display an interface (1212) for a second application in association with the element (531) based on the object pointing to the element (531). For example, the interface (1212) may be displayed in an upper area of ​​an object (530) that includes the element (531).

[0249] Although FIG. 12B illustrates an operation of performing input through a remote controller (1251), the present invention is not limited thereto. Identifiable inputs (e.g., input for an element (531), input for switching virtual space) in the wearable device (200) may be performed through at least one of a part of the user's body, an external device (e.g., a remote controller, another wearable device (e.g., a watch), a keyboard, a mouse), and / or voice input.

[0250] FIG. 13 illustrates an example of operation of a wearable device for displaying a visual object related to a second application within a second virtual space, according to one embodiment.

[0251] Referring to FIG. 13, in state (1301), the processor (410) can display a first portion (1311) of a second virtual space (1310) through a display (420). The processor (410) can display an object (530) within the first portion (1311) of the virtual space (1310). For example, the object (530) can correspond to the object (530) illustrated in FIG. 5A. For example, the second virtual space (1310) can be provided based on the execution of the first application.

[0252] According to one embodiment, the processor (410) may identify a second application running in a virtual space distinct from the second virtual space (1310) in the object (530). For example, the processor (410) may identify a second application running in the background while the second virtual space (1310) is displayed. The processor (410) may display an element (531) related to the second application within the object (530).

[0253] In one embodiment, the processor (410) may identify an input for removing an object (530). For example, the processor (410) may identify an input for removing a display of the object (530) from a first portion (1311) of the second virtual space (1310). For example, the processor (410) may identify an input for hiding a display of the object (530) within the second virtual space (1310).

[0254] For example, the processor (410) can change the display mode of the object (530). The processor (410) can set the display mode of the object (530) to one of a normal mode and a hidden mode. Based on changing the display mode of the object (530) from the normal mode to the hidden mode, the processor (410) can not display the object (530) within the second virtual space (1310). According to an embodiment, the processor (410) can display the object (530) within the second virtual space (1310) based on a specified action (or input).

[0255] According to one embodiment, the processor (410) may change the state of the wearable device (200) from state (1301) to state (1302) based on identifying an input to remove an object (530).

[0256] In state (1302), the processor (410) may not display the object (530) within the second virtual space (1310) based on an input to remove the object (530). The processor (410) may identify an element (531) included in the object (530). Based on identifying that the element (531) is not displayed together with the object (530), the processor (410) may display an interface (1315) for a second application corresponding to the element (531) in the first portion (1311) of the second virtual space (1310).

[0257] For example, the processor (410) may display an interface (1315) for the second application in the second virtual space (1310) (or a first portion (1311) of the second virtual space (1310)) based on identifying that an element (531) indicating that the second application is running is not displayed while the second virtual space (1310) is displayed.

[0258] FIG. 14 illustrates an example of operation of a wearable device for displaying a visual object related to a second application within a second virtual space, according to one embodiment.

[0259] Referring to FIG. 14, in state (1401), the processor (410) can display a first virtual space (1410) (or a portion of the virtual space (1410)). The processor (410) can display an interface (1411) related to a first application, an interface (1412) related to a second application, and an interface (1413) related to a third application within the virtual space (1410). The processor (410) can display an object (530) within the virtual space (1410). The object (530) can correspond to the object (530) of FIG. 5A.

[0260] For example, the processor (410) can identify an input for switching the first virtual space (1410) to the second virtual space (1420). The object (530) can include an element (1415) for executing a fourth application for providing the second virtual space (1420). The processor (410) can identify an input for switching the first virtual space (1410) to the second virtual space (1420) based on the input to the element (1415). The processor (410) can change the state of the wearable device (200) from state (1401) to state (1402) based on the input for switching the first virtual space (1410) to the second virtual space (1420).

[0261] In state (1402), the processor (410) may display a second virtual space (1420) (or a portion of the second virtual space (1420)) through the display (420) based on the execution of the fourth application. The processor (410) may display an object (530) within the second virtual space (1420). The processor (410) may maintain the display of the object (530) even when the virtual space displayed through the display (420) is changed.

[0262] For example, the processor (410) can identify an application that is executed while the second virtual space (1420) is displayed among the first application, the second application, and the third application executed in the first virtual space (1410). The processor (410) can identify that the first application and the second application are executed while the second virtual space (1420) is displayed. The processor (410) can display an element (531) indicating that the first application is executing on the object (530). The processor (410) can display an element (532) indicating that the second application is executing on the object (530).

[0263] According to one embodiment, the processor (410) may display the first virtual space (1410) based on identifying that execution of the fourth application providing the second virtual space (1420) has been stopped. The processor (410) may remove elements (531) and (532) from the object (530) within the first virtual space (1410). Based on removing elements (531) and (532) from the object (530), the processor (410) may display an interface (1411) related to the first application and an interface (1412) related to the second application. The processor (410) may display an interface (1413) related to the third application together with the interface (1411) and the interface (1412). For example, the processor (410) may change the state of the wearable device (200) from state (1402) to state (1401) based on identifying that execution of the fourth application providing the second virtual space (1420) has been stopped.

[0264] FIG. 15 illustrates an example of operation of a wearable device for displaying an application running within a first virtual space, according to one embodiment.

[0265] Referring to FIG. 15, in state (1501), the processor (410) can display a virtual space (1510) (or a portion of the virtual space (1510)) through a display (420). The processor (410) can display an element (1505) for displaying applications running in one or more virtual spaces including the virtual space (1510) within an object (530).

[0266] According to one embodiment, the processor (410) can identify an input to the element (1505). The processor (410) can change the state of the wearable device (200) from state (1501) to one of state (1502) and state (1503) depending on the setting value of the wearable device (200).

[0267] At state (1502), the processor (410) may display one or more visual objects (1520) within the virtual space (1510) based on identifying an input to the element (1505). For example, the one or more visual objects (1520) may include visual object (1520-1), visual object (1520-2), and visual object (1520-3). As an example, visual object (1520-1) may correspond to a visual object for providing at least one interface within the virtual space (1510). As an example, each of visual object (1520-2) and visual object (1520-3) may correspond to a visual object for providing at least one interface within another virtual space distinct from the virtual space (1510). For example, each of the one or more visual objects (1520) may display at least one interface for at least one running application. For example, a visual object (1520-1) may include at least one interface regarding at least one application running within a virtual space (1510) in which the visual object (1520-1) is displayed. For example, a visual object (1520-2) may include at least one interface regarding at least one application running within a virtual space in which the visual object (1520-2) is displayed. For example, a visual object (1520-3) may include at least one interface regarding at least one application running within a virtual space in which the visual object (1520-3) is displayed.

[0268] For example, a visual object (1520-1) may be used to provide at least one interface within a virtual space (1510). The visual object (1520-1) may be displayed at the top of one or more visual objects (1520).

[0269] According to an embodiment, the processor (410) may change a virtual space displayed through the display (420) to a virtual space corresponding to the visual object (1520-2) based on identifying an input for the visual object (1520-2). The processor (410) may display the visual object (1520-2) within the changed virtual space.

[0270] At state (1502), the processor (410) may, based on identifying an input to element (1505), display one or more interfaces (1530) relating to one or more applications running within the wearable device (200). For example, the processor (410) may identify applications running within the virtual space (1510), as well as applications running within another virtual space distinct from the virtual space (1510). The processor (410) may display one or more interfaces (1530) relating to one or more applications running within the wearable device (200). For example, applications corresponding to interface (1530-1), interface (1530-2), and interface (1530-3), respectively, may be running within the virtual space (1510). Although not shown, the processor (410) may display interfaces of applications running within another virtual space distinct from the virtual space (1510).

[0271] Although not shown, in some embodiments, the processor (410) may display interfaces for applications running on the wearable device (200) based on one of a stack, a list, a grid, a carousel, a slim list, and / or a vertical stack.

[0272] FIG. 16 illustrates an example of operation of a wearable device for displaying an application running within a first virtual space, according to one embodiment.

[0273] Referring to FIG. 16, in state (1601), the processor (410) may display a virtual space (1610) through the display (420). The processor (410) may display a visual object (1611) for displaying at least one interface for at least one application running within the virtual space (1610). For example, the visual object (1611) may include at least one area for displaying at least one interface. For example, the processor (410) may display an object (530). As an example, the object (530) may correspond to the object (530) of FIG. 5A.

[0274] For example, a visual object (1611) may include an interface (1615) for a first application. The processor (410) may display the interface (1615) within the visual object (1611) based on the execution of the first application. While the interface (1615) is displayed within the visual object (1611), the processor (410) may identify an input for execution of a second application. Based on the input for execution of the second application, the processor (410) may change the state of the wearable device (200) from state (1601) to state (1602).

[0275] In state (1602), the processor (410) may display an interface (1617) regarding the second application within the visual object (1611). For example, the processor (410) may display the interface (1617) on the area where the interface (1615) is placed in state (1601). The processor (410) may execute the first application in the background based on the execution of the second application. The processor (410) may remove the interface (1615) within the visual object (1611) based on the execution of the first application in the background. The processor (410) may display an element (1621) within the object (530) based on the execution of the first application in the background. The processor (410) may display an element (1621) indicating that the first application is executing in the background.

[0276] In one embodiment, the processor (410) can identify an input to an element (1621) while the element (1621) is displayed within the object (530). Based on the input to the element (1621), the processor (410) can change the state of the wearable device (200) from state (1602) to state (1603).

[0277] In state (1603), the processor (410) may display an interface (1625) based on an input to the element (1621). For example, the interface (1625) may be distinct from the interface (1615) displayed in state (1601). The processor (410) may display a reduced version of the interface (1625) within the virtual space (1610). For example, the interface (1625) may be displayed in conjunction with the element (1621). For example, the interface (1625) may be displayed as an overlay on the visual object (1611). The operation of the processor (410) for displaying the interface (1625) may correspond to the operation of the processor (410) described in FIG. 12B.

[0278] According to the above-described embodiment, the processor (410) can execute the first application in the background. Instead of displaying the interface (1615) on the visual object (1611) within the virtual space (1610), the processor (410) can display an element (1621) within the object (530). The processor (410) can display the element (1621) within the object (530) so that the user (510) can identify the execution information of the first application without interfering with the work performed in the virtual space (1610).

[0279] According to one embodiment, a wearable device may include a display, at least one sensor, a memory storing instructions and including one or more storage media, and at least one processor including a processing circuit. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify at least one application running within a first virtual space while the first virtual space is displayed. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify an input for running a first application within the first virtual space. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to switch the first virtual space to a second virtual space provided according to the first application based on the execution of the first application. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify, while the second virtual space is displayed, a second application running within the first virtual space among the at least one application. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify a motion of the wearable device while the wearable device displays a first portion of the second virtual space based on a first direction toward which a user's gaze is directed.The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display a visual object relating to the second application within a second portion of the second virtual space based on a second direction in which the user's gaze is directed, which direction is changed from the first direction in response to the motion of the wearable device.

[0280] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display an interface for the second application within a designated area within the second portion of the second virtual space based on identifying an input to the visual object.

[0281] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to perform a transition from the second virtual space to the first virtual space in which the second application is executed, based on identifying an input for the visual object. The at least one processor may cause the wearable device to display an interface relating to the second application within the first virtual space.

[0282] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display an object, on a designated area within the first portion of the second virtual space, that includes one or more elements relating to one or more applications running within at least one virtual space distinct from the second virtual space.

[0283] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify an input to an element of the one or more elements relating to the second application. The at least one processor may, based on identifying the input to the element, cause the at least one processor to display an interface relating to the second application within the first portion within the second virtual space.

[0284] According to one embodiment, the element may be displayed based on execution information of an action related to the second application.

[0285] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display an interface for the second application within the first portion within the second virtual space based on identifying an input for removing the object.

[0286] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to remove the element within the object displayed within the first virtual space based on identifying an input for transitioning from the second virtual space to the first virtual space.

[0287] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display at least one interface relating to the at least one application within the second portion of the second virtual space based on the second direction.

[0288] According to one embodiment, the first virtual space may be configured to include at least one area for executing the at least one application. The second virtual space may be configured to provide an interface for the first application.

[0289] According to one embodiment, a method of a wearable device may include an operation of identifying at least one application running in a first virtual space while the first virtual space is displayed. The method may include an operation of identifying an input for running a first application within the first virtual space. The method may include an operation of switching the first virtual space to a second virtual space provided according to the first application based on the execution of the first application. The method may include an operation of identifying, while the second virtual space is displayed, a second application running within the first virtual space among the at least one application. The method may include an operation of identifying a motion of the wearable device while displaying a first portion of the second virtual space based on a first direction in which the user's gaze is directed. The method may include an operation of displaying a visual object related to the second application within a second portion of the second virtual space based on a second direction in which the user's gaze is directed, which direction is changed from the first direction according to the motion of the wearable device.

[0290] In one embodiment, the method may include displaying an interface for the second application within a designated area within the second portion of the second virtual space based on identifying an input to the visual object.

[0291] In one embodiment, the method may include an operation of performing a transition from the second virtual space to the first virtual space in which the second application is executed, based on identifying an input for the visual object. The method may include an operation of displaying an interface related to the second application within the first virtual space.

[0292] According to one embodiment, the method may include displaying an object, on a designated area within the first portion of the second virtual space, that includes one or more elements relating to one or more applications running within one or more virtual spaces distinct from the second virtual space.

[0293] In one embodiment, the method may include an operation of identifying an input for an element related to the second application among the one or more elements. The method may include an operation of displaying an interface of the second application within the first portion within the second virtual space based on the identification of the input for the element.

[0294] According to one embodiment, the element may be displayed based on execution information of an action related to the second application.

[0295] In one embodiment, the method may include displaying an interface for the second application within the first portion within the second virtual space based on identifying an input for removing the object.

[0296] In one embodiment, the method may include an action of removing the element within the object displayed within the first virtual space based on identifying an input for transitioning from the second virtual space to the first virtual space.

[0297] According to one embodiment, the method may include displaying at least one interface relating to the at least one application within the second portion of the second virtual space based on the second direction.

[0298] According to one embodiment, a non-transitory computer readable storage medium may store one or more programs. The one or more programs may include instructions that, when executed by at least one processor of a wearable device having a display and at least one sensor, cause the wearable device to identify at least one application running within a first virtual space while the first virtual space is displayed. The one or more programs may include instructions that, when executed by the at least one processor of the wearable device, cause the wearable device to identify an input for running a first application within the first virtual space. The one or more programs may include instructions that, when executed by the at least one processor of the wearable device, cause the wearable device to transition the first virtual space to a second virtual space provided according to the first application based on the execution of the first application. The one or more programs may include instructions that, when executed by the at least one processor of the wearable device, cause the wearable device to identify, while the second virtual space is displayed, a second application running within the first virtual space among the at least one application. The one or more programs may include instructions that, when executed by the at least one processor of the wearable device, cause the wearable device to identify a motion of the wearable device while displaying a first portion of the second virtual space based on a first direction toward which a user's gaze is directed.The one or more programs may include instructions that, when executed by the at least one processor of the wearable device, cause the wearable device to display a visual object related to the second application within a second portion of the second virtual space based on a second direction in which the gaze of the user is directed, which direction is changed from the first direction according to the motion of the wearable device.

[0299] According to the above-described embodiment, when switching from a first virtual space to a second virtual space, there is an effect that the virtual space can be switched directly from the first virtual space to the second virtual space without going through the home virtual space. According to the above-described embodiment, in a multi-tasking environment, information about an application running in the first virtual space can be displayed within the second virtual space. There is an effect that information about an application running in the first virtual space is displayed while maintaining a screen for tasks related to the currently running application.

[0300] Electronic devices according to embodiments disclosed herein may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to embodiments disclosed herein are not limited to the aforementioned devices.

[0301] The embodiments of this document and the terminology used herein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among the phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0302] In one embodiment of this document, the term "module" used may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0303] One embodiment of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0304] According to one embodiment, the method according to one embodiment disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0305] According to one embodiment, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and arranged in other components. According to one embodiment, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to one embodiment, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In wearable devices, display; At least one sensor; A memory storing instructions and including one or more storage media; and comprising at least one processor including a processing circuit; The above instructions, when individually or collectively executed by the at least one processor, cause the wearable device to: While the first virtual space is displayed, identifying at least one application running within said first virtual space; Within the first virtual space, identify an input for executing a first application, Based on the execution of the first application, the first virtual space is converted into a second virtual space provided according to the first application, While the second virtual space is displayed, identify a second application running within the first virtual space among the at least one application; While displaying a first part of the second virtual space based on a first direction in which the user's gaze is directed, the motion of the wearable device is identified, Causing a visual object related to the second application to be displayed within a second part of the second virtual space based on a second direction in which the gaze of the user is directed, which is changed from the first direction according to the motion of the wearable device. Wearable devices.

2. In the first paragraph, when the instructions are individually or collectively executed by the at least one processor, the wearable device, Based on identifying an input to said visual object, causing an interface for said second application to be displayed within a designated area within said second portion of said second virtual space. Wearable devices.

3. In the first paragraph, when the instructions are individually or collectively executed by the at least one processor, the wearable device, Based on identifying the input for the visual object, performing a transition from the second virtual space to the first virtual space where the second application is executed; Causing an interface for the second application to be displayed within the first virtual space; Wearable devices.

4. In the first paragraph, when the instructions are individually or collectively executed by the at least one processor, the wearable device, Causing an object to be displayed, on a designated area within said first portion of said second virtual space, comprising one or more elements relating to one or more applications running within at least one virtual space distinct from said second virtual space; Wearable devices.

5. In the fourth paragraph, when the instructions are individually or collectively executed by the at least one processor, the wearable device, Identifying an input to an element relating to said second application among one or more of said elements, Based on identifying the input to said element, causing an interface for said second application to be displayed within said first portion within said second virtual space, Wearable devices.

6. In paragraph 4, the element is, Displayed based on the execution information of the action regarding the second application above, Wearable devices.

7. In the fourth paragraph, when the instructions are individually or collectively executed by the at least one processor, the wearable device, Based on identifying an input for removing said object, causing an interface for said second application to be displayed within said first portion within said second virtual space. Wearable devices.

8. In the fourth paragraph, when the instructions are individually or collectively executed by the at least one processor, the wearable device, Based on identifying an input for transitioning from the second virtual space to the first virtual space, causing the element to be removed from the object displayed within the first virtual space. Wearable devices.

9. In the first paragraph, when the instructions are individually or collectively executed by the at least one processor, the wearable device, Based on the second direction, causing at least one interface for the at least one application to be displayed within the second part of the second virtual space. Wearable devices.

10. In the first paragraph, the first virtual space is, configured to include at least one area for executing at least one application; The above second virtual space is, configured to provide an interface for the first application, Wearable devices.

11. In the method of a wearable device, An action for identifying at least one application running within the first virtual space while the first virtual space is displayed; An action for identifying an input for executing a first application within the first virtual space; An operation of switching the first virtual space to a second virtual space provided according to the first application based on the execution of the first application; An operation of identifying a second application running within the first virtual space, among the at least one application, while the second virtual space is displayed; An operation of identifying a motion of the wearable device while displaying a first portion of the second virtual space based on a first direction in which the user's gaze is directed; and An operation of displaying a visual object related to the second application within a second part of the second virtual space based on a second direction toward which the user's gaze is directed, which is changed from the first direction according to the motion of the wearable device. method.

12. In the 11th paragraph, the method, Further comprising an action of displaying an interface for the second application within a designated area within the second portion of the second virtual space, based on identifying an input for the visual object. method.

13. In the 11th paragraph, the method, An operation of performing a transition from the second virtual space to the first virtual space in which the second application is executed, based on identifying an input for the visual object; and Further comprising an action of displaying an interface for the second application within the first virtual space; method.

14. In the 11th paragraph, the method, Further comprising an action of displaying an object including one or more elements relating to one or more applications running in one or more virtual spaces distinct from the second virtual space, on a designated area within the first portion of the second virtual space. method.

15. A non-transitory computer readable storage medium storing one or more programs, wherein the one or more programs, when executed by at least one processor of a wearable device having a display and at least one sensor, While the first virtual space is displayed, identifying at least one application running within said first virtual space; Within the first virtual space, identify an input for executing a first application, Based on the execution of the first application, the first virtual space is converted into a second virtual space provided according to the first application, While the second virtual space is displayed, identify a second application running within the first virtual space among the at least one application; While displaying a first part of the second virtual space based on a first direction in which the user's gaze is directed, the motion of the wearable device is identified, Instructions for causing the wearable device to display a visual object related to the second application within a second portion of the second virtual space based on a second direction in which the gaze of the user is directed, the second direction being changed from the first direction according to the motion of the wearable device. Computer readable storage medium.

Images