Wearable device for executing one or more software applications on basis of location of wearable device, and method therefor

Abstract

A wearable device according to an embodiment comprises: one or more sensors; a display; a memory for storing instructions; and a processor for executing the instructions. The processor is configured to: define, on the basis of scanning for real space, an area including a plurality of regions; identify the location of the wearable device with respect to the area on the basis of sensor data acquired from the one or more sensors; and, on the basis of identifying that the location of the wearable device is included in a first region from among the plurality of regions included in the area, execute the software application related to the first region so as to display, on a display, a screen provided by the software application in order to provide virtual reality.

Description

Wearable device and method for executing one or more software applications based on the location of the wearable device

[0001] The present disclosure relates to a wearable device and a method for executing one or more software applications based on the location of the wearable device.

[0002] To provide an enhanced user experience, electronic devices are being developed that provide augmented reality (AR) services, which display computer-generated information in conjunction with external objects in the real world. These electronic devices may be wearable devices worn by the user. For example, these electronic devices may be AR glasses and / or head-mounted devices (HMDs).

[0003] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art related to the present disclosure.

[0004] According to one embodiment, a wearable device may include one or more sensors, a display, a memory for storing instructions, and at least one processor for executing the instructions. The instructions, when executed by the processor, may cause the wearable device to define a zone including a plurality of areas based on scanning of a physical space. The instructions, when executed by the processor, may cause the wearable device to identify a location of the wearable device relative to the zone based on sensor data acquired from the one or more sensors. The instructions, when executed by the processor, may cause the wearable device to execute a software application associated with the first region based on identifying that the location of the wearable device is included in a first region among the plurality of regions included in the zone, and to display a screen provided by the software application on the display to provide a virtual reality.

[0005] In one embodiment, a method of a wearable device may include an operation of defining a zone including a plurality of areas based on scanning of a real space. The method may include an operation of identifying a location of the wearable device with respect to the zone based on sensor data acquired from one or more sensors of the wearable device. The method may include an operation of executing a software application related to the first area based on identifying that the location of the wearable device is included in a first area among the plurality of areas included in the zone, and displaying a screen provided by the software application on a display of the wearable device to provide a virtual reality.

[0006] In one embodiment, a wearable device may include one or more sensors, a display, a memory for storing instructions, and at least one processor for executing the instructions. The at least one processor may be configured to, in response to an input related to the execution of a first software application among a plurality of software applications, identify a plurality of areas corresponding to at least one of the plurality of software applications and a zone including the plurality of areas based on sensor data related to external space obtained from the one or more sensors. The at least one processor may be configured to display a first screen provided by the first software application on the display based on identifying a location of the wearable device included in a first area corresponding to the first software application among the plurality of areas. The at least one processor may be configured to display a second screen for guiding movement to the first area based on identifying a location of the wearable device spaced apart from the first area.

[0007] In one embodiment, a method of a wearable device may include, in response to an input related to the execution of a first software application among a plurality of software applications, an operation of identifying a plurality of areas corresponding to at least one of the plurality of software applications and a zone including the plurality of areas based on sensor data related to external space obtained from one or more sensors of the wearable device. The method may include an operation of displaying a first screen provided by the first software application on a display of the wearable device based on identifying a location of the wearable device included in a first area corresponding to the first software application among the plurality of areas. The method may include an operation of displaying a second screen for guiding movement to the first area based on identifying a location of the wearable device spaced apart from the first area.

[0008] In one embodiment, a wearable device may include one or more sensors, a display, a memory for storing instructions, and at least one processor for executing the instructions. The at least one processor may be configured to obtain sensor data related to a location of the wearable device from the one or more sensors while displaying a first screen related to virtual reality or augmented reality. The at least one processor may be configured to display a virtual object on the display for adjusting a size of the first area within the area based on identifying a location of the wearable device within a zone comprising a plurality of zones spaced apart from a first area corresponding to the first screen and including the first area using the sensor data. The at least one processor may be configured to stop displaying the first screen on the display based on identifying a location of the wearable device spaced apart from the zone using the sensor data.

[0009] In one embodiment, a method of a wearable device may include an operation of acquiring sensor data related to a location of the wearable device from one or more sensors of the wearable device while displaying a first screen related to virtual reality or augmented reality. The method may include an operation of displaying a virtual object on a display of the wearable device for adjusting a size of the first area within the area based on identifying a location of the wearable device within a zone that includes a plurality of areas and is spaced apart from a first area corresponding to the first screen using the sensor data. The method may include an operation of ceasing to display the first screen on the display based on identifying a location of the wearable device spaced apart from the area using the sensor data.

[0010] FIG. 1 illustrates one embodiment of a wearable device that determines a zone and one or more areas within an external space.

[0011] FIG. 2 illustrates an example block diagram of a wearable device according to one embodiment.

[0012] FIG. 3 illustrates an example of a flowchart of a wearable device according to one embodiment.

[0013] FIG. 4 illustrates an example of a zone and multiple areas within an external space identified by a wearable device.

[0014] Figure 5 illustrates an example of a user interface (UI) displayed by a wearable device within an area.

[0015] FIG. 6 illustrates an example of a flowchart of a wearable device according to one embodiment.

[0016] FIGS. 7A and 7B illustrate an example of the operation of a wearable device for executing a software application based on a determined area within an external space.

[0017] FIG. 8 illustrates an example of a flowchart of a wearable device according to one embodiment.

[0018] Figures 9a, 9b, 9c, 9d and 9e illustrate exemplary states of a wearable device that displays a screen based on a location relative to an area.

[0019] FIGS. 10A and 10B illustrate exemplary states of a wearable device that displays a screen based on a location relative to an area.

[0020] FIG. 11A illustrates an example of a perspective view of a wearable device, according to one embodiment.

[0021] FIG. 11B illustrates an example of one or more hardware elements disposed within a wearable device, according to one embodiment.

[0022] FIGS. 12A and 12B illustrate an example of an appearance of a wearable device according to one embodiment.

[0023] Hereinafter, various embodiments of this document are described with reference to the attached drawings.

[0024] The various embodiments of this document and the terminology used therein are not intended to limit the technology described in this document to a specific embodiment, but should be understood to include various modifications, equivalents, and / or substitutes of the embodiment. In connection with the description of the drawings, similar reference numerals may be used for similar components. The singular expression may include plural expressions unless the context clearly indicates otherwise. In this document, expressions such as "A or B", "at least one of A and / or B", "A, B, or C", or "at least one of A, B, and / or C" may include all possible combinations of the items listed together. Expressions such as "first", "second", "first", or "second" may modify the corresponding components regardless of order or importance, and are only used to distinguish one component from another, but do not limit the corresponding components. When it is said that a component (e.g., a first component) is “(functionally or communicatively) connected” or “connected” to another component (e.g., a second component), said component may be directly connected to said other component, or may be connected via another component (e.g., a third component).

[0025] The term "module" as used in this document includes a unit composed of hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimal unit or portion thereof that performs one or more functions. For example, a module may be composed of an application-specific integrated circuit (ASIC).

[0026] FIG. 1 illustrates an embodiment of a wearable device (101) that determines a zone (130) and one or more areas within an external space. The wearable device (101) may include a head-mounted display (HMD) that is wearable on the head of a user (110). The wearable device (101) may be referred to as a head-mounted device (HMD), a headgear electronic device, a glasses-type electronic device, a video see-through (VST) or visible see-through device, an extended reality (XR) device, a virtual reality (VR) device, and / or an augmented reality (AR) device. Although the external appearance of the wearable device (101) in the form of glasses is illustrated, the embodiment is not limited thereto. An example of a hardware configuration included in the wearable device (101) is exemplarily described with reference to FIG. 2. An example of the structure of a wearable device (101) that can be worn on the head of a user (110) is described with reference to FIGS. 11A, 11B, 12A, and / or 12B. The wearable device (101) may be referred to as an electronic device. For example, the electronic device may be combined with an accessory (e.g., a strap) for attaching to the head of a user to form an HMD.

[0027] According to one embodiment, a wearable device (101) may perform functions related to augmented reality (AR) and / or mixed reality (MR). For example, when a user (110) wears the wearable device (101), the wearable device (101) may include at least one lens positioned adjacent to the user's (110) eyes. The wearable device (101) may combine ambient light passing through the lens with light emitted from a display of the wearable device (101). A display area of ​​the display may be formed within the lens through which the ambient light passes. Because the wearable device (101) combines the ambient light and the light emitted from the display, the user (110) may see an image that is a mixture of a real object recognized by the ambient light and a virtual object formed by the light emitted from the display. The augmented reality, mixed reality, and / or virtual reality described above may be referred to as extended reality (XR).

[0028] In one embodiment, a wearable device (101) may perform functions related to video see-through (VST) and / or virtual reality (VR). For example, when a user (110) wears the wearable device (101), the wearable device (101) may include a housing that covers the eyes of the user (110). The wearable device (101), in this state, may include a display disposed on a first side of the housing facing the eyes. The wearable device (101) may include a camera disposed on a second side opposite the first side. Using the camera, the wearable device (101) may acquire images and / or videos representing ambient light. The wearable device (101) can output the image and / or video within the display disposed on the first surface, thereby allowing the user (110) to recognize the ambient light through the display. The displaying area (or displaying region) (or active area or active region) of the display disposed on the first surface can be formed by one or more pixels included in the display. The wearable device (101) can synthesize a virtual object into the image and / or video output through the display, thereby allowing the user (110) to recognize the virtual object together with a real object recognized by the ambient light.

[0029] According to one embodiment, the wearable device (101) can identify or recognize a position (or location) and / or direction (or orientation) of the wearable device (101) based on images (and / or videos) obtained or acquired using a camera. Referring to FIG. 1, within an exemplary state (191) contained in an external space (e.g., a room) including external objects (121, 122, 123, 124, 125, 126), the wearable device (101) can obtain information about the external space using one or more cameras and / or one or more sensors. The information can include a geographic location (e.g., global positioning system (GPS) coordinates) of the external space identified from one or more sensors. The information can include images and / or videos of the external space identified from one or more cameras. The wearable device (101) can perform object recognition on an image and / or video to identify external objects (121, 122, 123, 124, 125, 126) included in the external space from the image and / or video.

[0030] The wearable device (101) may provide a user experience based on virtual reality and / or augmented reality while being worn by a user (110). For example, the wearable device (101) may display a virtual object (or imaginary object) that covers at least a portion of the field-of-view (FoV) of the user (110). If a real object is occluded by the virtual object within the field-of-view, the user (110) who does not recognize the real object may collide with the real object. In one embodiment, the wearable device (101) may identify a region (130) within the external space that has a relatively low collision probability based on the distribution (or positional relationship) of real objects (e.g., external objects (121, 122, 123, 124, 125, 126)) within the external space.

[0031] Referring to FIG. 1, a zone (130) may be determined based on a collision probability between a user (110) wearing a wearable device (101) and an actual object. For example, the collision probability within the zone (130) may be lower than the collision probability outside the zone (130). For example, the wearable device (101) may determine the zone (130) based on the floor of an external space. For example, a portion of the floor that is not occupied by an external object (123), such as a bookshelf, may be included in the zone (130). The wearable device (101) may insert a portion (e.g., the top surface of the desk) associated with the user (110) within the real object into a zone (130) based on identifying a real object of a specified type (category and / or class) including a sofa (e.g., external object (121)), a chair (e.g., external object (125)), and / or a desk (e.g., external object (126)). The zone (130) may be referred to as a safety region or safety zone, a guardian region, in terms of providing a collision-free virtual reality and / or augmented reality.

[0032] According to one embodiment, the wearable device (101) may divide the zone (130) into one or more areas. Referring to FIG. 1, exemplary zones (140) within the zone (130) defined by the wearable device (101) are illustrated. The wearable device (101) may determine a plurality of zones (140) within the zone (130) using actual objects included within the zone (130). For example, the wearable device (101) may determine a portion of the zone (130) that includes an external object (121) corresponding to a sofa as a first area (141). For example, the wearable device (101) may determine a portion of the zone (130) associated with external objects (125, 126) corresponding to a chair and a desk, respectively, as a second area (142). For example, the wearable device (101) may determine a portion of the zone (130) in which no real objects are placed as a third zone (143). The third zone (143) may be a portion of the zone (130) that is separated from all included real objects. The operation of the wearable device (101) for dividing zones (140) from the zone (130) is described with reference to FIGS. 3 and / or 4.

[0033] According to one embodiment, the wearable device (101) may provide virtual reality and / or augmented reality to a user (110) wearing the wearable device (101) by using the zone (130) and / or the areas (140). For example, when the location of the wearable device (101) and / or the user (110) located in any one of the areas (140) is identified, the wearable device (101) may recommend or guide the execution of one or more software applications related to the area included in the location to the user (110). The embodiment is not limited thereto, and the wearable device (101) may execute one or more software applications corresponding to the area included in the location. Based on the execution of the one or more software applications, the wearable device (101) may output a screen and / or audio based on virtual reality and / or augmented reality to the user (110).

[0034] As described above, according to one embodiment, the wearable device (101) can divide the zone (130) for providing virtual reality and / or augmented reality, and determine zones (140) corresponding to different software applications. Based on the location of the wearable device (101) with respect to the determined zones (140), the wearable device (101) can execute one or more software applications or recommend them to the user (110). The location of at least one external object included in the zone (130) can be used to determine the zones (140). When the wearable device (101) identifies the wearable device (101) and / or the user (110) that has moved into the zone (130), the wearable device (101) can guide the user to move to one of the zones (140) corresponding to the software application executed by the wearable device (101). The wearable device (101) can identify a region including the location of the wearable device (101) and / or the user (110) within the zone (130) by comparing the location with each of the regions (140). The wearable device (101) can execute at least one software application corresponding to the identified region and / or recommend it to the user (110).

[0035] Hereinafter, with reference to FIG. 2, an example of a hardware configuration of a wearable device (101) for determining a zone (130) and / or areas (140) is described.

[0036] FIG. 2 illustrates an example block diagram of a wearable device (101) according to one embodiment. The wearable device (101) described with reference to FIG. 1 may include at least one of the hardware components of the wearable device (101) distinguished by different blocks in FIG. 2.

[0037] Referring to FIG. 2, a wearable device (101) according to one embodiment may include at least one of a processor (210), a memory (215), a display (220), a camera (225), a sensor (230), and a communication circuit (235). The processor (210), the memory (215), the display (220), the camera (225), the sensor (230), and / or the communication circuit (235) may be electrically and / or operably coupled with each other by an electronic component such as a communication bus (202). The type and / or number of hardware components included in the wearable device (101) is not limited to those illustrated in FIG. 2. For example, the wearable device (101) may include only some of the hardware components illustrated in FIG. 2.

[0038] According to one embodiment, the processor (210) of the wearable device (101) may include a hardware component for processing data based on a plurality of instructions. The hardware component for processing data may include, for example, an arithmetic and logic unit (ALU), a field programmable gate array (FPGA), a central processing unit (CPU), and / or an application processor (AP). In one embodiment, the wearable device (101) may include one or more processors. The processor (210) may have a multi-core processor structure, such as a dual core, a quad core, a hexa core, and / or an octa core.

[0039] The memory (215) of the wearable device (101) according to one embodiment may include a hardware component for storing data and / or instructions input to and / or output from the processor (210). The memory (215) may include, for example, volatile memory such as random-access memory (RAM) and / or non-volatile memory such as read-only memory (ROM). The volatile memory may include, for example, at least one of dynamic RAM (DRAM), static RAM (SRAM), cache RAM, and pseudo SRAM (PSRAM). The non-volatile memory may include, for example, at least one of programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, hard disk, compact disc, and embedded multi media card (eMMC). In one embodiment, the memory (215) may be referred to as storage.

[0040] In one embodiment, the display (220) of the wearable device (101) can output visualized information (e.g., screen (510) of FIG. 5 , screens (710, 711, 712) of FIGS. 7A and / or 7B , screen (910) of FIGS. 9A to 9E , and / or screen (1010) of FIGS. 10A to 10B ) to a user of the wearable device (101) (e.g., user (110) of FIG. 1 ). For example, the display (220) can be configured to visualize information provided from a processor (210) that includes circuitry such as a graphic processing unit (GPU). The display (220) can include a flexible display, a flat panel display (FPD), and / or electronic paper. The FPD may include a liquid crystal display (LCD), a plasma display panel (PDP), and / or one or more light emitting diodes (LEDs). The LEDs may include organic LEDs (OLEDs). The embodiment is not limited thereto, and for example, if the wearable device (101) includes a lens for transmitting external light (or ambient light), the display (220) may include a projector (or projection assembly) for projecting light onto the lens. In one embodiment, the display (220) may be referred to as a display panel and / or a display module.

[0041] In one embodiment, the camera (225) of the wearable device (101) may include optical sensors (e.g., a charged coupled device (CCD) sensor, a complementary metal oxide semiconductor (CMOS) sensor) that generate electrical signals representing the color and / or brightness of light. The camera (225) may be referred to as an image sensor and may be included in the sensor (230) of FIG. 2. The plurality of optical sensors included in the camera (225) may be arranged in the form of a two-dimensional array. The camera (225) may acquire electrical signals of each of the plurality of optical sensors substantially simultaneously, and generate two-dimensional frame data corresponding to light reaching the optical sensors of the two-dimensional array. For example, photographic data captured using the camera (225) may mean one (a) two-dimensional frame data acquired from the camera (225). For example, video data captured using a camera (225) may mean a sequence of a plurality of two-dimensional frame data acquired from the camera (225) according to a frame rate. The camera (225) may further include a flash light that is positioned toward the direction in which the camera (225) receives light and outputs light toward the direction.

[0042] According to one embodiment, the wearable device (101) may include a plurality of cameras, for example, cameras (225), arranged facing different directions. Referring to FIG. 2, the camera (225) included in the wearable device (101) may include a gaze tracking camera (225-1) and / or an outward camera (225-2). The gaze tracking camera (225-1) may be arranged toward at least one of the two eyes of a user wearing the wearable device (101). The processor (210) may identify the direction of the user's gaze using images and / or videos acquired from the gaze tracking camera (225-1). The gaze tracking camera (225-1) may include an infrared (IR) sensor. The gaze tracking camera (225-1) may be referred to as an eye sensor, a gaze tracker, and / or an eye tracker.

[0043] Referring to FIG. 2, the external camera (225-2) may be positioned to face the front of a user wearing the wearable device (101) (e.g., a direction in which both eyes may face). Using images and / or videos acquired from the external camera (225-2), the processor (210) may identify external objects (e.g., external objects 121, 122, 123, 124, 125, 126 of FIG. 1). The embodiment is not limited thereto, and the processor (210) may identify a position, shape, and / or gesture (e.g., a hand gesture) of a hand based on images and / or videos acquired from the external camera (225-2).

[0044] According to one embodiment, a sensor (230) of a wearable device (101) may generate electrical information from non-electronic information related to the wearable device (101), which may be processed and / or stored by a processor (210) and / or a memory (215) of the wearable device (101). The information may be referred to as sensor data. The sensor (230) may include a global positioning system (GPS) sensor for detecting a geographic location of the wearable device (101), an image sensor, an audio sensor (e.g., a microphone and / or a microphone array), an ambient light sensor, and / or a time-of-flight (ToF) sensor (or a ToF camera).

[0045] In one embodiment, the sensor (230) may include an inertial measurement unit (IMU) for detecting physical motion of the wearable device (101). An acceleration sensor, a geomagnetic sensor, a gravity sensor, or any combination thereof may be referred to as an IMU. The acceleration sensor may output electrical signals representing gravitational acceleration and / or accelerations of each of a plurality of axes (e.g., x-axis, y-axis, and z-axis) that are perpendicular to each other and based on a designated origin of the wearable device (101). The gyro sensor may output electrical signals representing angular velocities (e.g., roll, pitch, and / or yaw) of the plurality of axes. The gyro sensor may be referred to as an angular velocity sensor. The geomagnetic sensor may output an electrical signal representing the magnitude of a magnetic field formed in the wearable device (101) along each of a plurality of axes (e.g., the x-axis, the y-axis, and / or the z-axis). For example, the acceleration sensor, the gyro sensor, and / or the geomagnetic sensor may repeatedly output sensor data including accelerations, angular velocities, and / or magnitudes of magnetic fields of the plurality of axes based on a specified period (e.g., 1 millisecond).

[0046] In one embodiment, the communication circuit (235) of the wearable device (101) may include circuitry for supporting transmission and / or reception of electrical signals between the wearable device (101) and an external electronic device. The communication circuit (235) may include, for example, at least one of a modem, an antenna, and an optical / electronic (O / E) converter. The communication circuit (235) may support transmission and / or reception of electrical signals based on various types of protocols, such as Ethernet, a local area network (LAN), a wide area network (WAN), wireless fidelity (WiFi), Bluetooth, Bluetooth low energy (BLE), ZigBee, long term evolution (LTE), 5G new radio (NR), 6G and / or above-6G. In one embodiment, the communication circuit (235) may be referred to as a communication processor and / or a communication module.

[0047] According to one embodiment, one or more instructions (or commands) representing data to be processed, calculations to be performed, and / or operations to be performed by the processor (210) of the wearable device (101) may be stored in the memory (215) of the wearable device (101). A set of one or more instructions may be referred to as a program, firmware, an operating system, a process, a routine, a sub-routine, and / or a software application (hereinafter, “application”). For example, the wearable device (101) and / or the processor (210) may perform at least one of the operations of FIG. 3, FIG. 6, and / or FIG. 8 when a set of a plurality of instructions distributed in the form of an operating system, firmware, driver, program, and / or application is executed. Hereinafter, the fact that an application is installed in a wearable device (101) may mean that one or more instructions provided in the form of an application are stored in a memory (215), and that the one or more applications are stored in a format executable by the processor (210) (e.g., a file having an extension specified by the operating system of the wearable device (101)). For example, an application may include a program and / or a library related to a service provided to a user.

[0048] Referring to FIG. 2, programs installed in the wearable device (101) may be classified into any one of different layers, including an application layer (240), a framework layer (250), and / or a hardware abstraction layer (HAL) (280), based on the target. For example, programs (e.g., modules or drivers) designed to target the hardware of the wearable device (101) (e.g., the display (220), the camera (225), the sensor (230), and / or the communication circuit (235)) may be classified within the hardware abstraction layer (280). The framework layer (250) may be referred to as an XR framework layer in terms of including one or more programs for providing an XR (extended reality) service. For example, the layers illustrated in FIG. 2 may be logically (or for convenience of explanation) separated, and may not mean that the address space of the memory (215) is separated by the layers.

[0049] For example, programs classified as a framework layer (250) (e.g., a position tracker (271), a spatial recognizer (272), a gesture tracker (273), an eye tracker (274), and / or a face tracker (275)) may generate information that is processed by other programs classified as a hardware abstraction layer (280) and / or an application layer (240). Programs classified as a framework layer (250) may provide an application programming interface (API) that is executable (or callable) based on other programs.

[0050] For example, within the application layer (240), programs designed to target users of the wearable device (101) may be classified. The application layer (240) may include an extended reality (XR) system user interface (UI) (241) and / or an XR application (242). For example, programs (e.g., software applications) classified into the application layer (240) may call an application programming interface (API) to cause the execution of functions supported by programs classified into the framework layer (250).

[0051] For example, the wearable device (101) may display one or more visual objects on the display (220) for performing interaction with a user for using a virtual space based on the execution of the XR system UI (241). A visual object may refer to an object that can be deployed within a screen for transmitting and / or interacting with information, such as text, an image, an icon, a video, a button, a checkbox, a radio button, a text box, a slider, and / or a table. A visual object may be referred to as a visual guide, a virtual object, a visual element, a UI element, a view object, and / or a view element. The wearable device (101) may provide the user with functions available within the virtual space based on the execution of the XR system UI (241).

[0052] Referring to FIG. 2, a lightweight renderer (243) and / or an XR plug-in (244) are illustrated to be included within the XR system UI (241), but are not limited thereto. For example, based on the XR system UI (241), the processor (210) may execute a lightweight renderer (243) and / or an XR plug-in (244) within the framework layer (250).

[0053] For example, the wearable device (101) may obtain resources (e.g., APIs, system processes, and / or libraries) used to define, create, and / or execute a rendering pipeline that allows partial changes based on the execution of a lightweight renderer (243). The lightweight renderer (243) may be referred to as a lightweight render pipeline in terms of defining a rendering pipeline that allows partial changes. The lightweight renderer (243) may include a renderer built prior to the execution of a software application (e.g., a prebuilt renderer). For example, the wearable device (101) may obtain resources (e.g., APIs, system processes, and / or libraries) used to define, create, and / or execute an entire rendering pipeline based on the execution of an XR plug-in (244). The XR plugin (244) can be referred to as an open XR native client from the perspective of defining (or setting up) the entire rendering pipeline.

[0054] For example, the wearable device (101) may display a screen representing at least a portion of a virtual space on the display (220) based on the execution of the XR application (242). The XR plug-in (244-1) included in the XR application (242) may include instructions that support functions similar to those of the XR plug-in (244) of the XR system UI (241). Descriptions of the XR plug-in (244-1) that overlap with those of the XR plug-in (244) may be omitted. The wearable device (101) may cause the execution of the virtual space manager (251) based on the execution of the XR application (242).

[0055] According to one embodiment, the wearable device (101) may provide a virtual space service based on the execution of the virtual space manager (251). For example, the virtual space manager (251) may include a platform for supporting the virtual space service. The wearable device (101) may identify a virtual space formed based on the user's location indicated by data acquired through the sensor (230) based on the execution of the virtual space manager (251), and may display at least a portion of the virtual space on the display (220). The virtual space manager (251) may be referred to as a composition presentation manager (CPM).

[0056] For example, the virtual space manager (251) may include a runtime service (252). As an example, the runtime service (252) may be referred to as an OpenXR runtime module (or an OpenXR runtime program). The wearable device (101) may execute at least one of a user's pose prediction function, a frame timing function, and / or a spatial input function based on the execution of the runtime service (252). As an example, the wearable device (101) may perform rendering for a virtual space service to the user based on the execution of the runtime service (252). For example, a function related to a virtual space, executable by the application layer (240), may be supported based on the execution of the runtime service (252).

[0057] For example, the virtual space manager (251) may include a pass-through manager (253). Based on the execution of the pass-through manager (253), the wearable device (101) may display a screen representing a virtual space on the display (220), while superimposing another screen representing an actual space obtained through an external camera (225-2) on at least a portion of the screen.

[0058] For example, the virtual space manager (251) may include an input manager (254). The wearable device (101) may identify data (e.g., sensor data) obtained by executing one or more programs included in the recognition service layer (270) based on the execution of the input manager (254). The wearable device (101) may use the obtained data to identify user input related to the wearable device (101). The user input may be related to a motion (e.g., hand gesture), gaze, and / or speech of the user identified by the sensor (230).

[0059] For example, the perception abstract layer (260) can be used for data exchange between the virtual space manager (251) and the perception service layer (270). From the perspective of being used for data exchange between the virtual space manager (251) and the perception service layer (270), the perception abstract layer (260) can be referred to as an interface. For example, the perception abstract layer (260) can be referenced as OpenPX. The perception abstract layer (260) can be used for a perception client and a perception service.

[0060] According to one embodiment, the recognition service layer (270) may include one or more programs for processing data acquired from a sensor (230) and / or a camera (225). The one or more programs may include at least one of a position tracker (271), a space recognizer (272), a gesture tracker (273), an eye tracker (274), a face tracker (275), and / or an interface processor (276). The type and / or number of the one or more programs included in the recognition service layer (270) are not limited to those illustrated in FIG. 2.

[0061] For example, the wearable device (101) can identify the pose of the wearable device (101) using the sensor (230) based on the execution of the position tracker (271). The wearable device (101) can identify the 6 degrees of freedom pose (6 dof pose) of the wearable device (101) using data acquired using the external camera (225-2) and / or IMU (e.g., a gyro sensor, an acceleration sensor, and / or a geomagnetic sensor) based on the execution of the position tracker (271). The position tracker (271) may be referred to as a head tracking (HeT) module (or head tracker, a head tracking program).

[0062] For example, the wearable device (101) may obtain information for providing a three-dimensional virtual space corresponding to the surrounding environment (e.g., external space) of the wearable device (101) (or the user of the wearable device (101)) based on the execution of the space recognizer (272). The wearable device (101) may reconstruct the surrounding environment of the wearable device (101) in three dimensions using data obtained using the external camera (225-2) based on the execution of the space recognizer (272). The wearable device (101) may identify at least one of a plane, a slope, and stairs based on the surrounding environment of the wearable device (101) reconstructed in three dimensions based on the execution of the space recognizer (272). The space recognizer (272) may be referred to as a scene understanding (SU) module (or a scene understanding program).

[0063] For example, the wearable device (101) can identify (or recognize) a pose and / or gesture of a hand of a user of the wearable device (101) based on the execution of the gesture tracker (273). As an example, the wearable device (101) can identify a pose and / or gesture of a hand of a user using data acquired from an external camera (225-2) based on the execution of the gesture tracker (273). As an example, the wearable device (101) can identify a pose and / or gesture of a hand of a user based on data (or images) acquired using the external camera (225-2) based on the execution of the gesture tracker (273). The gesture tracker (273) may be referred to as a hand tracking (HaT) module (or hand tracking program) and / or a gesture tracking module.

[0064] For example, the wearable device (101) can identify (or track) eye movements of a user of the wearable device (101) based on the execution of the gaze tracker (274). As an example, the wearable device (101) can identify eye movements of the user using data acquired from the gaze tracking camera (225-1) based on the execution of the gaze tracker (274). The gaze tracker (274) may be referred to as an eye tracking (ET) module (or eye tracking program) and / or a gaze tracking module.

[0065] For example, the recognition service layer (270) of the wearable device (101) may further include a face tracker (275) for tracking the user's face. For example, the wearable device (101) may identify (or track) the movement of the user's face and / or the user's expression based on the execution of the face tracker (275). The wearable device (101) may estimate the user's expression based on the movement of the user's face based on the execution of the face tracker (275). As an example, the wearable device (101) may identify the movement of the user's face and / or the user's expression based on data (e.g., an image) acquired using a sensor (230) (e.g., an image sensor directed at at least a portion of the user's face) based on the execution of the face tracker (275).

[0066] According to one embodiment, the processor (210) of the wearable device (101) can identify the location of the wearable device (101) within an external space, as indicated by information obtained from the camera (225) and / or the sensor (230). Based on the location, the wearable device (101) can provide at least a portion of a virtual space mapped (or linked) with the external space. For example, the wearable device (101) can determine the location of the wearable device (101) within the virtual space based on the location. The determined location within the virtual space can correspond to a reference point (e.g., a view point) that indicates a portion of the virtual space to be displayed through the display (220). The processor (210) of the wearable device (101) can execute one or more instructions to identify the location within the external space by executing the location tracker (271) of FIG. 2.

[0067] According to one embodiment, the processor (210) of the wearable device (101) may obtain information related to an external space including the wearable device (101) based on the execution of the space recognizer (272). The operation of the processor (210) to obtain information related to the external space may be referred to as space recognition and / or space scanning. Based on the space recognition, the processor (210) may identify at least one real object included in the external space (e.g., external objects (121, 122, 123, 124, 125, 126) of FIG. 1). The processor (210) may determine an area (e.g., area (130) of FIG. 1) for providing virtual reality and / or augmented reality using the information obtained based on the space recognition and / or space scan.

[0068] The processor (210) may generate or determine different regions (e.g., regions (140) of FIG. 1) within the zone based on one or more software applications (e.g., software applications classified as application layers (240)) installed on the wearable device (101). The regions may be associated with, correspond to, and / or connected to at least one software application. The processor (210) may divide one or more regions within the zone based on properties (and / or characteristics) of each of the software applications. For example, the processor (210) may divide at least one region based on a size (e.g., size of the activity area), width, height, area, and / or radius set by the software application. The regions within the zone may not overlap each other. Alternatively, the regions within the zone may be spaced apart from each other.

[0069] A processor (210) that divides a plurality of areas within a zone may execute a function related to an area when the wearable device (101) and / or a user is located within one of the areas. The function may include an operation of recommending at least one software application related to the area among a plurality of software applications installed in the wearable device (101). The function may include an operation of executing at least one software application related to the area. The function may include an operation of providing virtual reality and / or augmented reality provided from the at least one software application. The function may include an operation for providing an immersive user experience, such as active noise cancellation (ANC).

[0070] While the wearable device (101) is executing a function related to an area within which the wearable device (101) is located, the processor (210) may identify movement of the wearable device (101) outside the area. In response to the movement, the processor (210) may perform an operation for re-executing the function. For example, the processor (210) may guide the user wearing the wearable device (101) to move back into the area. For example, the processor (210) may at least temporarily suspend execution of the function related to the area.

[0071] As described above, according to one embodiment, the processor (210) of the wearable device (101) can divide an area that matches the user experience provided by the software application within the area formed for the user's safety based on virtual reality and / or augmented reality. If multiple software applications are installed on the wearable device (101), the processor (210) can determine an area corresponding to at least one of the multiple software applications from the area. By utilizing the area included in the area, the processor (210) can improve the user experience related to virtual reality and / or augmented reality. For example, even if the user moves out of the area, if the user is included in the area including the area, the processor (210) can maintain execution of the software application corresponding to the area and can continue to provide a user experience based on the software application.

[0072] Hereinafter, with reference to FIG. 3, an exemplary operation performed by a processor (210) of a wearable device (101) according to one embodiment is described.

[0073] FIG. 3 illustrates an example flowchart of a wearable device according to one embodiment. The wearable device (101) of FIGS. 1 and 2 and / or the processor (210) of FIG. 2 may perform at least one of the operations of FIG. 3. The operations of FIG. 3 may be related to the operations of the processor executing the spatial recognizer (272) of FIG. 2.

[0074] Referring to FIG. 3, in operation (310), according to one embodiment, a processor of a wearable device may recognize an external space. The processor may identify a shape of the external space based on an image and / or a video acquired from a camera (e.g., the camera 225 and / or an external camera 225-2 of FIG. 2). The processor may identify at least one external object included in the external space based on the image and / or the video. Identifying the external object by the processor may include an operation of identifying a location of the external object within the external space and / or a size of a space occupied by the external object. The processor may use a sensor (230) (e.g., an IMU) to determine a location and / or orientation of the wearable device within the external space.

[0075] Referring to FIG. 3, in operation (320), a processor of a wearable device according to an embodiment may identify whether zone information corresponding to an external space is stored. The zone information may be matched to the external space recognized based on operation (310). The zone information may be stored in a memory of the wearable device (e.g., memory 215 of FIG. 2). The zone information may include an identifier of the external space corresponding to the zone information (e.g., ID and / or GPS coordinates of the external space). The zone information may include data representing one or more areas divided from the zone. When the zone information stored in the wearable device is identified (320 - Yes), the processor may perform operation (360) based on the zone information. When the zone information is not identified from the memory of the wearable device (320 - No), the processor may perform operation (330).

[0076] Referring to FIG. 3, in operation (330), according to one embodiment, a processor of a wearable device may determine a region for virtual reality and / or augmented reality based on a distribution of external objects included in an external space. The external objects of operation (330) may be identified based on object recognition of images and / or videos output from a camera. Based on operation (330), the processor may determine at least a portion of the external space, in which the probability of collision with the external objects is relatively low, as the region of operation (330). The region of operation (330) may include region (130) of FIG. 1. For example, the processor may identify a region formed to provide virtual reality and / or augmented reality based on data from sensors and / or cameras. Based on operation (330), the processor of the wearable device may perform scanning of an actual space (physical). Based on the above scanning, the processor can define a zone containing multiple areas.

[0077] Referring to FIG. 3, within operation (340), a processor of a wearable device according to one embodiment may identify an input for editing a region. Operation (340) may be optionally performed by the processor. Based on operation (340), the processor may visualize the region determined by operation (330) on a display (e.g., display (220) of FIG. 2 ). For example, the processor may display a virtual object on the display representing the region. Based on a user's hand gesture, gaze, and / or speech associated with the virtual object, the processor may identify the input of operation (340). In response to the input, the processor may change the region determined by operation (330).

[0078] Referring to FIG. 3, in operation (350), according to one embodiment, a processor of a wearable device may divide a zone into different regions based on categories of software applications. The software applications of operation (350) may be installed in the memory of the wearable device. Based on the categories identified by the properties of each of the software applications, the processor may divide the zone into regions corresponding to each of the categories. Each of the regions divided based on operation (350) may correspond to each of the categories. The processor may store region information representing the regions divided based on operation (350) in the memory of the wearable device.

[0079] Referring to FIG. 3, in operation (360), according to one embodiment, a processor of a wearable device may recommend or provide at least one software application based on the segmented regions. The processor may perform operation (360) based on the regions indicated by the zone information of operation (320). The segmented regions of operation (360) may be identified based on operation (350). Based on operation (360), the processor may guide the user to one or more software applications included in a category corresponding to each of the regions. Based on operation (360), the processor may execute one or more software applications related to one of the regions where the wearable device is located. Based on the execution of the one or more software applications, the processor may provide a user experience related to virtual reality and / or augmented reality.

[0080] Hereinafter, with reference to FIG. 4, exemplary operations of a wearable device for determining zones and / or areas within an external space by performing the operations of FIG. 3 are described.

[0081] FIG. 4 illustrates an example of a zone (130) and a plurality of areas (140) within an external space identified by a wearable device (101). The wearable device (101) of FIGS. 1 and 2 and / or the processor (210) of FIG. 2 may perform the operation of the wearable device (101) of FIG. 4. The operation of the wearable device (101) described with reference to FIG. 4 may be related to at least one of the operations of FIG. 3.

[0082] Referring to FIG. 4, within an exemplary external space including external objects (121, 122, 123, 124, 125, 126), the wearable device (101) may determine a zone (130) based on a space scan. The zone (130) may be related to a positional relationship of the external objects (121, 122, 123, 124, 125, 126) within the external space, a position of the wearable device (101) within the external space (e.g., an elevation of the wearable device (101) from the floor of the external space), and / or a posture of a user (110) wearing the wearable device (101).

[0083] For example, within the city of FIG. 4 (as shown in), the wearable device (101) can determine a zone (130) within an external space using the position and / or elevation of the wearable device (101) based on the posture of a standing user (110). When a user (110) wearing the wearable device (101) sits on an external object (121), such as a sofa, the wearable device (101) can determine a zone (130) having a size and / or shape related to the sitting posture of the user (110). When a change in the posture of the user (110) is identified, the wearable device (101) can change the size, shape, and / or position of the zone (130) within the external space based on the changed posture. The embodiment is not limited thereto, and the wearable device (101) may change the shape and / or size of the zone (130) based on user input for editing the zone (130).

[0084] Referring to FIG. 4, examples of locations and / or sizes of regions (140) determined within a region (130) are illustrated. A first region (141) may be determined based on a portion of the region (130) associated with an external object (121), such as a sofa. A second region (142) may be determined as a portion of the region (130) in which external objects (125, 126), corresponding to a desk and a chair, are located. A third region (143) may be determined as a portion of an external space separated from all external objects. A portion of a boundary of the third region (143) may be adjacent to or overlapping with a boundary of the region (130). Another portion of a boundary of the third region (143) may be adjacent to another region (e.g., the first region (141) and / or the second region (142)). The wearable device (101) can store information representing a combination of zones (130) and areas (140) (e.g., zone information of FIG. 3).

[0085] The wearable device (101) may determine at least one software application corresponding to each of the regions (140) based on the shapes of the regions (140) and / or external objects included in each of the regions (140). For example, the wearable device (101) may match a software application requiring high-intensity physical activity (e.g., a game and / or a health application) to a third region (143) having the largest size among the regions (140). Each of the software applications installed in the wearable device (101) may include an attribute indicating a minimum size of a region corresponding to the software application. Based on the attribute, the processor may match a software application having a minimum size smaller than the size of the third region (143) with the third region (143).

[0086] For example, for a second area (142) associated with a desk (e.g., external object (126)) and / or a chair (e.g., external object (125)), the wearable device (101) may match software applications (e.g., a word processor, a co-work application, and / or a video conferencing application) associated with the desk and / or chair. If multiple desks are identified within the external space, the wearable device (101) may match different software applications to each of the multiple desks based on the sizes and / or heights of the multiple desks. The connection relationship between the multiple desks and the software applications may be changed by a user (110) wearing the wearable device (101).

[0087] For example, for a first area (141) associated with a sofa (e.g., an external object (121)), the wearable device (101) can match a software application for playing media content (e.g., a video). The result of matching the areas (140) with at least one software application can be stored in the wearable device (101). Based on the history of executing the software application in at least one of the areas (140), the wearable device (101) can match the areas (140) with at least one application.

[0088] According to one embodiment, the wearable device (101) can identify a location of the wearable device (101) and / or the user (110) within an external space based on images and / or videos acquired from a camera (e.g., the camera (225) and / or the external camera (225-2) of FIG. 2). For example, the wearable device (101) that has identified a location of a specific body part (e.g., the head and / or hand) of the user (110) can identify a specific area including the location among the areas (140). For example, by comparing the location of the wearable device (101) within an external space and each of the areas (140), the wearable device (101) can identify one area corresponding to the wearable device (101). When the wearable device (101) is connected to a remote controller (e.g., a remote controller that can be grabbed by the hand of the user (110)), the wearable device (101) can compare the location of the remote controller with the areas (140) to identify an area in which the wearable device (101) and / or the user (110) are located. Based on the identified area, the wearable device (101) can recommend or execute at least one software application.

[0089] Below, an exemplary operation of a wearable device (101) located in one of the areas (140) guiding the execution of at least one software application is described.

[0090] Fig. 5 illustrates an example of a user interface (UI) displayed by a wearable device (101) within a zone (130). The wearable device (101) of Figs. 1 and 2 and / or the processor (210) of Fig. 2 may perform the operations of the wearable device (101) of Fig. 5. The operations of the wearable device (101) described with reference to Fig. 5 may be related to at least one of the operations of Fig. 3.

[0091] Referring to FIG. 5, an exemplary state (501) of a wearable device (101) providing a list of software applications installed in the wearable device (101) is illustrated. Referring to FIG. 5, within a screen (510) displayed on a display (e.g., display (220) of FIG. 2), the wearable device (101) may display a virtual object (520) including a list of software applications. The virtual object (520) may have a two-dimensional rectangular shape. The virtual object (520) may be referred to as a panel. The screen (510) including the list of software applications may be referred to as a home screen (or launcher screen).

[0092] The wearable device (101) can display a screen (510) based on virtual reality and / or augmented reality. For example, the wearable device (101) can display an image and / or a virtual object (e.g., a background object) based on virtual reality on another part of the screen (510) that is distinct from the virtual object (520). For example, the wearable device (101) can display an image and / or a video (e.g., a video capturing an external environment) acquired from a camera (e.g., an external camera (225-2) of FIG. 2) on another part of the screen (510) that is distinct from the virtual object (520). Using the image and / or video acquired from the camera, the wearable device (101) can provide a user experience based on VST.

[0093] On the virtual object (520), the wearable device (101) may display icons (531, 532, 533, 534) corresponding to each of a plurality of software applications installed on the wearable device (101). Although four icons (531, 532, 533, 534) corresponding to four software applications are illustrated as an example, the embodiment is not limited thereto. The icons (531, 532, 533, 534) may include two-dimensional images provided by the software applications. The embodiment is not limited thereto, and the icons (531, 532, 533, 534) may have a three-dimensional shape, such as a rectangular parallelepiped.

[0094] Referring to FIG. 5, within an exemplary state (501) that identifies a zone (130) and zones (140) included in the zone (130), the wearable device (101) can determine which zone among the zones (140) corresponds to the wearable device (101). For example, the wearable device (101) can identify or determine which zone among the zones (140) is located where the wearable device (101) and / or the user (110) wearing the wearable device (101) is located.

[0095] Referring to FIG. 5, an exemplary state (501) is illustrated in which a user (110) wearing a wearable device (101) is located in a first area (141). In the state (501) in which the location of the wearable device (101) included in the first area (141) is identified, the wearable device (101) may search for or identify one or more software applications included in a category associated with the first area (141). The wearable device (101) may determine one or more software applications associated with the first area (141) based on at least one of the size of the first area (141), the category of each of the software applications, and / or an external object (121) included in the first area (141). The wearable device (101) may guide the user (110) to the determined one or more software applications.

[0096] For example, within a state (501) of providing a list of software applications based on a screen (510), the wearable device (101) may display a visual object (540) highlighting at least one software application associated with the first area (141). For example, the wearable device (101) may display a visual object (540) including designated text, such as “recommended,” as an overlay on an icon (533) representing the software application associated with the first area (141). The visual object (540) may be referred to as a badge and / or indicator for the icon (533).

[0097] Within a state (501) of displaying a screen (510) including a list of software applications, the wearable device (101) may receive an input related to any one of the icons (531, 532, 533, 534) corresponding to each of the software applications. The input may be identified based on a gaze of the user (110) toward any one of the icons (531, 532, 533, 534), a hand gesture (e.g., a pinch gesture and / or a pointing gesture) related to any one of the icons (531, 532, 533, 534), and / or an utterance related to any one of the icons (531, 532, 533, 534). In response to the input, the wearable device (101) may execute a software application corresponding to the icon related to the input. The wearable device (101) can display a screen provided by the software application on the display.

[0098] As described above, according to one embodiment, the wearable device (101) can guide the execution of one or more software applications based on the current location of the wearable device (101) and / or the user (110) (e.g., the user (110) wearing the wearable device (101)). For example, the wearable device (101) can identify the one or more software applications by comparing the current location with a plurality of areas (140) included in the zone (130). In one embodiment, the wearable device (101) that has received an input for executing a software application can execute the software application based on the area associated with the software application. For example, the wearable device (101) can execute the software application based on whether the wearable device (101) and / or the user (110) is located in a specific area within the zone (130) that is suitable for executing the software application.

[0099] Hereinafter, with reference to FIG. 6, the operation of a wearable device (101) that executes a software application based on the current location of the wearable device (101) and / or the user (110) is described.

[0100] FIG. 6 illustrates an example flowchart of a wearable device according to one embodiment. The wearable device (101) of FIGS. 1 and 2 and / or the processor (210) of FIG. 2 may perform at least one of the operations of FIG. 6. The operations of FIG. 6 may be related to at least one of the operations of FIG. 3.

[0101] Referring to FIG. 6, in operation (610), a processor of a wearable device according to one embodiment may receive an input for executing a software application. The processor may identify the input of operation (610) based on a gaze, hand gesture, speech, or a combination thereof of a user wearing the wearable device, recognized based on data from a camera (e.g., camera (225) of FIG. 2) and / or a sensor (e.g., sensor (230) of FIG. 2). Based on receiving the input of operation (610), the processor may perform operation (620).

[0102] Referring to FIG. 6, in operation (620), a processor of a wearable device according to one embodiment may identify whether the wearable device is located within an area corresponding to a software application. The processor may perform operation (620) using the identified location of the wearable device based on data from a camera and / or sensor. The embodiment is not limited thereto, and the processor may perform operation (620) based on the location of a body part (e.g., head, hands, and / or feet) of a user wearing the wearable device and / or the location of a remote controller connected to the wearable device.

[0103] The processor may identify an area corresponding to the software application based on the properties of the software application identified by the input. The area may be included within an area (e.g., area (130) of FIG. 1) identified to provide virtual reality and / or augmented reality, such as areas (140) of FIG. 1. The processor may compare the location of the wearable device with the area to identify whether the wearable device is located in the area. If the wearable device is located within the area corresponding to the software application (620—Yes), the processor may perform operation (640). If the wearable device is located outside the area of ​​operation (620—No), the processor may perform operation (630).

[0104] Referring to FIG. 6, in operation (630), according to one embodiment, a processor of a wearable device may display a virtual object for guiding an area on a display (e.g., display (220) of FIG. 2). The processor may perform operation (630) based on augmented reality (or mixed reality) among virtual reality or augmented reality. For example, the processor may display a virtual object having a position relative to at least a portion of the display corresponding to the area within an external space visible through the display. The virtual object may have the form of a virtual fence for indicating an area for executing the software application of operation (610). An exemplary form of the virtual object of operation (630) is exemplarily described with reference to FIG. 7B. Referring to FIG. 6, in a state where the virtual object of operation (630) is displayed, the processor may perform operation (620) to monitor whether the wearable device enters an area guided by the virtual object.

[0105] Referring to FIG. 6, in operation (640), according to one embodiment, a processor of a wearable device may execute a software application to display a screen based on virtual reality and / or augmented reality. If the current location of the wearable device falls within an area corresponding to the software application associated with the input of operation (610), the processor may execute the software application. The processor may display, on the display, a screen provided by the software application and based on virtual reality and / or augmented reality.

[0106] As described above, according to one embodiment, a wearable device can conditionally execute a software application using the location of the wearable device and / or a user wearing the wearable device. For example, if the location falls within an area suitable for executing the software application, the processor can execute the software application. Hereinafter, with reference to FIGS. 7A and / or 7B , an exemplary UI displayed by a wearable device performing the operation of FIG. 6 will be described.

[0107] FIGS. 7A and 7B illustrate an example of an operation of a wearable device (101) for executing a software application based on a determined area within an external space. The wearable device (101) of FIGS. 1 and 2 and / or the processor (210) of FIG. 2 may perform the operation of the wearable device (101) of FIGS. 7A and / or 7B. The operation of the wearable device (101) described with reference to FIGS. 7A and / or 7B may be related to at least one of the operations of FIGS. 3 and / or 6.

[0108] Referring to FIG. 7A, an exemplary state (701) of a wearable device (101) that identifies an input related to the execution of a software application is illustrated. The input may be identified by a gaze, a hand gesture, and / or a speech to any one of the icons (531, 532, 533, 534) of FIG. 5. In response to the input, the wearable device (101) may acquire data related to an external space from a sensor (e.g., sensor (230) of FIG. 2) and / or a camera (e.g., camera (225) of FIG. 2). Based on the data, the wearable device (101) may identify a zone (130) included in the external space and a plurality of zones (140) included in the zone (130). The processor may identify a specific zone among the zones (140) that is suitable for the execution of a software application corresponding to the input.

[0109] Referring to FIG. 7A, in a state (701) of receiving an input for executing a software application related to a third area (143) among the areas (140), the wearable device (101) may compare the location of the wearable device (101) and / or the user (110) with the third area (143). Based on identifying the location of the wearable device (101) and / or the user (110) included in the third area (143), the wearable device (101) may display a first screen provided by the software application on a display (e.g., the display (220) of FIG. 2). For example, when the wearable device (101) and / or the user (110) is located in the third area (143), the processor may directly execute the software application based on the input to obtain the first screen to be displayed on the display.

[0110] When the wearable device (101) and / or the user (110) is spaced from the third area (143) corresponding to the software application, the wearable device (101) may display a screen (710) to guide movement to the third area (143). In the exemplary state (701) of FIG. 7A, based on identifying the location of the wearable device (101) and / or the user (110) spaced from the third area (143) (e.g., a location within the first area (141)), the wearable device (101) may display a screen (710) to guide movement to the third area (143). The wearable device (101) may display a screen (710) based on augmented reality.

[0111] Referring to an exemplary screen (710) of FIG. 7A, the wearable device (101) may display a virtual object (e.g., virtual objects (720, 730)) superimposed on a portion of the screen (710) related to the third area (143) within the screen (710). The wearable device (101) displaying the screen (710) including the virtual object (720) may guide a user (110) wearing the wearable device (101) to move to the third area (143) defined by the virtual object (720). The virtual object (720) may have a form of a line drawn on the floor of an external space visible through the screen (710).

[0112] For example, the virtual object (720) may have the shape of a boundary line of the third area (143). According to one embodiment, the wearable device (101) may determine whether the external space to be displayed through the screen (710) includes the third area (143) and / or a floor corresponding to the third area (143). If at least a portion of the floor corresponding to the third area (143) is visible through the screen (710), the wearable device (101) may display at least a portion of the virtual object (720) having the shape of a boundary line of the floor (or an outline line of the floor).

[0113] In one embodiment, the wearable device (101) may display a virtual object (730) in the form of a three-dimensional icon corresponding to a software application within the screen (710) together with the virtual object (720). Referring to FIG. 7A, the virtual object (730) is exemplarily illustrated as a rectangular parallelepiped (or cube) including at least one face on which a two-dimensional image (e.g., an icon of a software application) is placed, but the embodiment is not limited thereto. Within the screen (710), the virtual object (730) may float on a third region (143).

[0114] While displaying the screen (710), the wearable device (101) can identify the location of the wearable device (101) and / or the user (110) with respect to the third area (143). For example, the wearable device (101) can identify movement of the wearable device (101) and / or the user (110) into the third area (143). In response to the movement, the wearable device (101) can display another screen, different from the screen (710), provided from a software application corresponding to the virtual object (730) within the screen (710).

[0115] The shape of the virtual object (720) displayed in the screen (710) to guide movement to the third area (143) is not limited to the shape of the two-dimensional line drawn on the floor of FIG. 7A. Referring to FIG. 7B, different shapes of the visual object (720) for guiding the shape and / or position of the third area (143) are illustrated. As in the screen (710) of FIG. 7B, the wearable device (101) may display a virtual object (720) in the shape of a two-dimensional line (e.g., a dashed line) drawn on the floor shown through the screen (710). As in the screen (711) of FIG. 7B, the wearable device (101) may display a virtual object (720) in the shape of a fence (or wall) extending in the height (e.g., in the +z-axis direction) from the two-dimensional line drawn on the floor.

[0116] Referring to FIG. 7B, the wearable device (101) can gradually reduce transparency of a virtual object (720) displayed on the screen (711) according to altitude. The embodiment is not limited thereto, and the wearable device (101) can gradually change the color of the virtual object (720) displayed on the screen (711) according to altitude (e.g., gradient). For example, the color of the virtual object (720) can gradually change from a color (e.g., representative color) of the virtual object (730) and / or a software application corresponding to the virtual object (730) to another color (e.g., gradient). At least a portion of the virtual object (720) within the screen (712) can be transparent.

[0117] Referring to FIG. 7B, a screen (712) including a virtual object (720) for guiding the shape and / or position of a third area (143) is exemplarily illustrated. The virtual object (720) can guide the third area (143) based on a plurality of dots. Through the plurality of dots, the user (110) of the wearable device (101) can view the virtual object (720) for the external space and the third area (143) visible through the screen (712). Based on the virtual object (720) having the shape of a virtual fence, the wearable device (101) can visualize the third area (143). The wearable device (101) can use the virtual object (720) to cause the user (110) to move inside the virtual fence.

[0118] As described above, according to one embodiment, the wearable device (101) may execute one of a plurality of software applications based on the current location of the wearable device (101) and / or the user (110). Based on the execution of the software application, the wearable device (101) may display a screen for virtual reality and / or augmented reality. If the area corresponding to the software application (e.g., the third area (143) of FIG. 7A) includes the current location of the wearable device (101) and / or the user (110), the wearable device (101) may display a screen provided from the software application. If the above area (e.g., the third area (143) of FIG. 7A) is outside the current location of the wearable device (101) and / or the user (110), the wearable device (101) may display a virtual object (720) to visualize the area based on augmented reality.

[0119] Hereinafter, with reference to FIG. 8, an exemplary operation of a wearable device (101) executing a software application corresponding to a specific area within a specific area is described.

[0120] FIG. 8 illustrates an example flowchart of a wearable device according to one embodiment. The wearable device (101) of FIGS. 1 and 2 and / or the processor (210) of FIG. 2 may perform at least one of the operations of FIG. 8. The operations of FIG. 8 may be related to the operations of the processor executing the spatial recognizer (272) of FIG. 2.

[0121] Referring to FIG. 8, in operation (810), a processor of a wearable device according to one embodiment may detect a location of the wearable device relative to an area while displaying a first screen for virtual reality and / or augmented reality based on execution of a software application. The area of ​​operation (810) may be associated with the software application of operation (810) among areas (e.g., areas (140) of FIG. 1) included within an area formed to provide virtual reality and / or augmented reality (e.g., area (130) of FIG. 1). The processor may track or identify a location of the wearable device to be compared to an area using a camera (e.g., camera (225) and / or external camera (225-2) of FIG. 2) and / or a sensor (e.g., sensor (230) of FIG. 2).

[0122] Referring to FIG. 8, within operation (820), a processor of a wearable device according to one embodiment may identify whether the wearable device has moved out of an area. If the detected location based on operation (810) is away from an area associated with a software application (820 - Yes), the processor may perform operation (830). If the detected location based on operation (810) is included in an area corresponding to the software application (820 - No), the processor may continue to detect or identify the location of the wearable device based on operation (810).

[0123] Referring to FIG. 8, in operation (830), according to one embodiment, a processor of a wearable device may display a second screen based on augmented reality. The second screen may be provided by the software application of operation (810) to at least partially maintain the user experience of the first screen of operation (810). The processor may use the second screen based on augmented reality to notify a user wearing the wearable device of the status of the wearable device moved out of the area. In one embodiment, the processor of the wearable device may display a virtual object for changing the execution state of the software application of operation (810) in the second screen. Based on an input related to the virtual object, the processor may change the execution state. The execution state of the software application that can be switched based on the virtual object may include at least one of an active state, a paused state, or a terminated state.

[0124] Referring to FIG. 8, in operation (840), a processor of a wearable device according to one embodiment may identify whether the wearable device has moved out of a zone including an area. The zone of operation (840) may be formed within an external space to safely provide virtual reality and / or augmented reality. The zone of operation (840) may include zone (130) of FIG. 1. If the wearable device has moved out of the zone (840 - Yes), the processor may perform operation (850). If the wearable device is included within the zone (840 - No), the processor may perform operation (860).

[0125] Referring to FIG. 8, in operation (850), according to one embodiment, a processor of a wearable device may stop execution of a software application based on a response sensitivity for a zone. The response sensitivity may be a sensitivity used to determine whether to stop providing virtual reality and / or augmented reality in response to a motion of the wearable device and / or the user moving out of the zone. For example, at least one of a threshold distance between a boundary of the zone and the wearable device (or the user), a threshold period of time of movement out of the zone, and / or a threshold frequency of movement out of the zone may be adjusted by the response sensitivity. If the wearable device ㅁ; moves out of the zone exceeding at least one of the threshold distance, threshold period, and / or threshold frequency described above, the processor may stop execution of the software application based on operation (850).

[0126] In one embodiment, the response sensitivity of operation (850) may be related to the category of the software application (e.g., the software application of operation (810)) being executed by the wearable device. For example, when executing a software application that requires a rapid action from the user, such as a game, the response sensitivity of operation (850) may be set relatively sensitively. For example, at least one of the threshold distance, the threshold period, and / or the threshold frequency may be decreased. In another example, when executing a software application that requires relatively few actions, such as a word processor, the response sensitivity of operation (850) may be relaxed. For example, at least one of the threshold distance, the threshold period, and / or the threshold frequency may be increased. A processor that has suspended execution of a software application based on operation (850) may display a virtual object for resuming execution of the software application, and may receive an input from the user to resume execution of the software application through the virtual object.

[0127] Referring to FIG. 8, in operation (860), according to one embodiment, a processor of a wearable device may display a virtual object for switching the state of a software application. The state of operation (860) may be an execution state of the software application, and may include at least one of an active state, a stopped state, or a terminated state. The virtual object of operation (860) may have the form of a dialog box.

[0128] Referring to FIG. 8, in operation (870), a processor of a wearable device according to one embodiment may change a state of a software application based on an input related to a virtual object. In one embodiment displaying a virtual object in the form of a dialog box, the processor may perform operation (870) based on an input indicating a selection of one of a plurality of buttons included in the dialog box. In response to the input of operation (870), the processor may continue execution of the software application, or at least temporarily suspend it.

[0129] Below, with reference to FIGS. 9A to 9E, an exemplary UI displayed by a wearable device performing the operation of FIG. 8 is described.

[0130] FIGS. 9A, 9B, 9C, 9D, and 9E illustrate exemplary states (901, 902, 903, 904, and 905) of a wearable device (101) displaying a screen (910) based on a location relative to an area. The wearable device (101) of FIGS. 1 and 2 and / or the processor (210) of FIG. 2 may perform at least one of the operations of FIGS. 9A to 9E. The operations of the wearable device (101) of FIGS. 9A to 9E may be related to the operations described with reference to FIGS. 3, 6, and / or 8.

[0131] Referring to FIGS. 9A to 9E , different states (901, 902, 903, 904, 905) of a wearable device (101) executing a software application (e.g., a game application) based on a third area (143) within a zone (130) are illustrated. The state (901) of FIG. 9A may be an exemplary state of the wearable device (101) performing the operation (810) of FIG. 8 . Within the exemplary state (901) of FIG. 9A , the wearable device (101) may display a screen (910) based on virtual reality on a display (e.g., the display (220) of FIG. 2 ). Within the screen (910), the wearable device (101) may display a foreground object (e.g., a virtual object (920)) floating within a virtual space together with a background object that completely obscures the external environment.

[0132] Within a state (901) related to virtual reality, a user (110) wearing a wearable device (101) can view a screen (910) that completely obscures the external environment. The wearable device (101) can maintain providing the screen (910) related to virtual reality within the state (901) while the location of the wearable device (101) and / or the user (110), which is identified based on data from a camera (e.g., camera (225) of FIG. 2) and / or a sensor (e.g., sensor (230) of FIG. 2), is maintained within the third region (143). Within the state (901) providing the screen (910) that obscures the external environment, the wearable device (101) can activate noise cancellation (e.g., ANC) or activate a do-not-disturb mode.

[0133] Within the state (901), while displaying the screen (910), the wearable device (101) can use data from sensors and / or cameras to identify whether the wearable device (101) and / or the user (110) moves out of the third area (143). The third area (143) can be determined based on the category of the software application providing the screen (910).

[0134] Referring to FIG. 9B, an exemplary state (902) in which the user (110) moves out of the third area (143) is illustrated. When switching from the state (901) of FIG. 9A to the state (902) of FIG. 9B, the processor may identify movement of the wearable device (101) out of the third area (143) based on data from the camera and / or sensor. In the state (902) in which the location of the wearable device (101) and / or the user (110) is identified as being away from the third area (143), the wearable device (101) may display a virtual object (921) on the display for controlling the execution state of a software application. When the wearable device (101) and / or the user (110) are moved out of the third area (143) and within the zone (130) in a state (902), the wearable device (101) may perform the operation (860) of FIG. 8 to display a virtual object (921). For example, the virtual object (921) of FIG. 9b may be included in the virtual object of the operation (860) of FIG. 8.

[0135] Referring to FIG. 9B, the wearable device (101) that has identified the location of the wearable device (101) and / or the user (110) that has moved out of the third area (143) may execute a software application that was being executed based on virtual reality based on augmented reality. Within the state (902), the wearable device (101) may display a screen (910) based on the augmented reality. Within the screen (910) within the state (902), the wearable device (101) may display a screen (910) that at least partially includes an image and / or video of the external environment. For example, within the screen (910), a background object related to the virtual reality may be removed, and a foreground object (e.g., a virtual object (920)) may continue to be displayed. As described above, a wearable device (101) that switches from a state (901) related to virtual reality to a state (902) related to augmented reality may, when noise cancellation is activated in the state (901), deactivate (or pause) the noise cancellation by switching to the state (902).

[0136] Referring to FIG. 9B, through a screen (910) based on augmented reality, the wearable device (101) can inform the user (110) of the location of the wearable device (101) and / or the user (110) that has moved out of the third area (143). Within the state (902), when the user (110) moves back into the third area (143), the wearable device (101) can identify the location of the wearable device (101) and / or the user (110) that has moved into the third area (143). In response to identifying the location, the wearable device (101) can switch back from the state (902) to the state (901) of FIG. 9A (e.g., a state that displays a screen (910) that completely obscures the external environment based on virtual reality). When noise cancellation is paused by switching from state (901) to state (902), the wearable device (101) can resume noise cancellation by switching back to state (901).

[0137] Within a state (902) in which the location of the wearable device (101) and / or the user (110) included within the zone (130) is identified, the wearable device (101) may display a virtual object (921) for switching the state of the software application while continuing to execute the software application. In one embodiment, whether or not to display the virtual object (921) may be changed by a setting value input by the user (110). Within the virtual object (921), the wearable device (101) may display text (e.g., “Do you want to continue executing the App?”) for guiding the switching of the state. Within the virtual object (921), the wearable device (101) may display buttons (922, 923) for receiving an input for switching the state.

[0138] Based on an input indicating selection of button (923), the wearable device (101) may at least temporarily suspend execution of the software application. For example, in response to an input to button (923), the wearable device (101) may stop displaying the virtual object (720) and display an icon representing the software application (e.g., the virtual object (730) of FIG. 7A) having a location corresponding to the third region (143), such as state (701) of FIG. 7A. Along with the icon, the wearable device (101) may display text, badges, and / or indicators indicating a state of the software application (e.g., a stage of a game application).

[0139] Based on an input indicating selection of a button (922), the wearable device (101) can maintain execution of a software application that was running within a state (902). For example, in response to the input, the wearable device (101) can switch from a state (902) of FIG. 9B to a state (903) of FIG. 9C. Within the state (903), the wearable device (101) can resume providing a virtual reality-based screen (910), similar to the state (901) of FIG. 9A. Within the state (903) moved outside the third region (143), the wearable device (101) providing the virtual reality-based screen (910) can adjust the boundary of the third region (143). For example, the wearable device (101) can add, within a state (903), a region (930) including the location of the wearable device (101) and / or the user (110) to the third region (143). Within the state (903), the wearable device (101) can compare the location of the wearable device (101) and / or the user (110) with the third region (143) combined with the region (930) to display a screen (910) based on virtual reality and / or augmented reality.

[0140] Although the operation of the wearable device (101) to change the third area (143) based on an input indicating selection of a button (922) is described, the embodiment is not limited thereto. For example, while displaying a screen based on a game application (e.g., screen (910) of FIG. 9A), if the number of times the wearable device (101) and / or the user (110) moves out of the third area (143) exceeds a specified number of times, the wearable device (101) may change the third area (143) based on the movement pattern of the wearable device (101) and / or the user (110) moving out of the third area (143). Conversely, when the wearable device (101) and / or the user (110) moves within a portion of the third area (143) while displaying a screen based on a game application (e.g., screen (910) of FIG. 9A), the wearable device (101) may reduce the size and / or shape of the third area (143).

[0141] After the shape and / or size of the third region (143) is changed (903), the wearable device (101) can change information indicating the changed third region (143) (e.g., zone information of the operation (320) of FIG. 3). Based on the change (or update) of the information, the third region (143) further including the region (930) can be used to provide a game application. For example, within the state (902) of FIG. 9b, the wearable device (101) can change the shape and / or size of the third region (143) based on an input related to a button (922) of a virtual object (921).

[0142] Referring to FIG. 9d, a state (904) is illustrated in which the wearable device (101) and / or the user (110) is moved out of the zone (130). Unlike the state (902) in which the wearable device (101) was moved within the zone (130), the wearable device (101) that has identified the wearable device (101) and / or the user (110) as being moved out of the zone (130) may switch to the state (904) of FIG. 9d and stop execution of a software application related to virtual reality or augmented reality. The state (904) of FIG. 9d may be an exemplary state of the wearable device (101) that has performed the operation (850) of FIG. 8.

[0143] The wearable device (101) may display a virtual object (940) within the screen (910) indicating that the execution of the software application has been stopped. Within the virtual object (940), the wearable device (101) may display text (e.g., “Move into the area”) to guide the movement of the wearable device (101) and / or the user (110) to the area (130) and / or the third area (143) associated with the software application. Within the state (904) in which the execution of the software application has been stopped, the wearable device (101) may stop displaying virtual objects (e.g., the virtual object (920) and / or the background object of FIG. 9A) associated with the software application. For example, within the state (904), the wearable device (101) may not display any virtual objects other than the virtual object (940). Although not shown, within state (904), a user (110) wearing a wearable device (101) can view images and / or videos corresponding to external space through a portion of the screen (910) that is distinct from the virtual object (940).

[0144] Although the operation of the wearable device (101) to suspend execution of a software application based on movement of the wearable device (101) and / or the user (110) has been described, the embodiment is not limited thereto. While executing a software application based on a third area (143) within a zone (130), the wearable device (101) may at least temporarily suspend execution of the software application based on a change in the state of the third area (143).

[0145] Referring to Fig. 9e, a state (905) in which a dog (950) moves into the third area (143) is illustrated. Although the operation of the wearable device (101) based on an animal such as a dog (950) is described, the embodiment is not limited thereto. For example, the wearable device (101) may perform the operation described below with reference to Fig. 9e based on an external object such as a robot vacuum cleaner and / or a person (a person distinct from the user (110) wearing the wearable device (101). For example, the wearable device (101) may perform the operation described below with reference to Fig. 9e based on identifying a change in the state of the third area (143) identified at the timing when the third area (143) was formed based on spatial recognition.

[0146] As in the exemplary state (905) of FIG. 9E, the wearable device (101) that has identified an external object entering the third area (143) may at least temporarily stop executing a software application based on the third area (143), or execute a software application based on augmented reality. Referring to FIG. 9E, by executing a software application based on augmented reality, the wearable device (101) may display a screen (910) in which a dog (950) is visible. Within the screen (910), the wearable device (101) may display a virtual object (921) for controlling the execution state of the software application. The operation of the wearable device (101) with respect to the virtual object (921) of FIG. 9E may be performed similarly to the operation of the wearable device (101) described with reference to FIG. 9B and / or FIG. 9C.

[0147] Although exemplary operations of a wearable device (101) executing a software application, such as a game application, within the third area (143) have been described, the embodiments are not limited thereto. Below, exemplary operations of a wearable device (101) executing a collaboration application (or video conferencing application) are described with reference to FIGS. 10A and 10B .

[0148] FIGS. 10A and 10B illustrate exemplary states of a wearable device (101) that displays a screen based on a location relative to an area. The wearable device (101) of FIGS. 1 and 2 and / or the processor (210) of FIG. 2 may perform the operations of the wearable device (101) described with reference to FIGS. 10A and / or 10B. The operations of the wearable device (101) of FIGS. 10A and / or 10B may be related to the operations of FIGS. 3, 6, and / or 8.

[0149] Referring to FIGS. 10A and / or 10B , different states (1001, 1002) of a wearable device (101) executing a software application (e.g., a collaboration application) based on a second area (142) within a zone (130) are illustrated. Within the state (1001) of FIG. 10A , the wearable device (101) may display a screen (1010) related to virtual reality based on the operation (810) of FIG. 8 . Within the screen (1010), the wearable device (101) may display at least a portion of a virtual space (e.g., a video conference room) provided by the collaboration application. For example, within the screen (1010), the wearable device (101) may display avatars (1011, 1012, 1013) representing other users logged in via one or more external electronic devices. Along with foreground objects such as avatars (1011, 1012, 1013), the wearable device (101) can display background objects that obscure the external environment.

[0150] Within state (1001), the wearable device (101) can perform noise cancellation to block interference from the external environment. Within state (1001), the wearable device (101) can adjust the brightness of the display covering the user's (110) eyes to assist the user (110) in focusing on the virtual reality for a relatively long period of time.

[0151] Referring to Fig. 10b, an exemplary state (1002) after the state (1001) of Fig. 10a is illustrated. As a user (110) wearing the wearable device (101) moves out of the second area (142), the state of the wearable device (101) may switch from the state (1001) of Fig. 10a to the state (1002) of Fig. 10b. Within the state (1002) of Fig. 10b, the wearable device (101) that has identified the location of the wearable device (101) and / or the user (110) that has moved out of the second area (142) may display a virtual object (1020) for switching the execution state of the collaborative application. Within the state (1002), the wearable device (101) may stop noise cancellation that was being executed based on the collaborative application. Within state (1002), the wearable device (101) may stop displaying a screen based on virtual reality (e.g., screen (1010) of FIG. 10A) and display images and / or videos of the external environment based on augmented reality and / or VST. Based on the images and / or videos, the user (110) of the wearable device (101) may view real objects in the external space (e.g., external objects (123) such as a bookshelf).

[0152] Within the state (1002) of FIG. 10b, which is switched from the state (1001) of FIG. 10a, the wearable device (101) may suspend or maintain the execution state of the collaboration application. When maintaining the execution of the collaboration application, the wearable device (101) may continue to output audio corresponding to the virtual space (e.g., a video conference room) provided by the collaboration application to the user (110) through the speaker. For example, within the state (1002), the user (110) may hear the speech of other users (e.g., users corresponding to avatars (1011, 1012, 1013)) logged into the virtual space through the wearable device (101).

[0153] Referring to FIG. 10B, within a screen (1010), the wearable device (101) may display a virtual object (1020) including buttons (1021, 1022, 1023) for adjusting the status of a collaboration application. A virtual object (1020) in the form of a dialog box is illustrated as an example, but the embodiment is not limited thereto. Within the virtual object (1020), the wearable device (101) may display a button (1021) for adjusting the status of a user (110) (e.g., away status) within a virtual space (e.g., video conference room) provided by the collaboration application. In response to an input indicating selection of the button (1021), the wearable device (101) may change the status of the user (110) to a status associated with the button (1021) based on the execution of the collaboration application.

[0154] Referring to FIG. 10b, within the virtual object (1020), the wearable device (101) may display a button (1022) for at least temporarily suspending execution of the collaboration application. The button (1022) may include designated text (e.g., “Stop”) guiding the suspending of the collaboration application. In response to an input associated with the button (1022), the wearable device (101) may suspend execution of the collaboration application. Referring to FIG. 10b, within the virtual object (1020), the wearable device (101) may display a button (1023) for adjusting the visibility of the virtual object (1020). While the button (1023) is illustrated to include text such as “Cancel,” the embodiment is not limited thereto. In response to an input to the button (1023), the wearable device (101) may stop displaying the virtual object (1020) within the screen (1010).

[0155] Hereinafter, exemplary appearances of the wearable devices described with reference to FIGS. 1 to 10B are illustrated with reference to FIGS. 11A, 11B, 12A, and / or 12B. The wearable device (1100) of FIG. 11A and / or 11B and / or the wearable device (1200) of FIG. 12A and / or 12B may be an example of the wearable device (101) of FIG. 1.

[0156] FIG. 11A illustrates an example of a perspective view of a wearable device according to one embodiment. According to one embodiment, the wearable device (1100) may have the form of glasses that are wearable on a body part of a user (e.g., the head). The wearable device (1100) may include a head-mounted display (HMD). For example, the housing of the wearable device (1100) may include a flexible material, such as rubber and / or silicone, that is configured to fit closely to a portion of the user's head (e.g., a portion of the face surrounding both eyes). For example, the housing of the wearable device (1100) may include one or more straps that are capable of being twined around the user's head, and / or one or more temples that are detachably attachable to the ears of the head.

[0157] Referring to FIG. 11A, according to one embodiment, a wearable device (1100) may include at least one display (1150) and a frame (1100) supporting at least one display (1150).

[0158] According to one embodiment, a wearable device (1100) can be worn on a part of a user's body. The wearable device (1100) can provide augmented reality (AR), virtual reality (VR), or mixed reality (MR) that combines augmented reality and virtual reality to a user wearing the wearable device (1100). For example, the wearable device (1100) can display a virtual reality image provided from at least one optical device (1182, 1184) of FIG. 11B on at least one display (1150) in response to a user's designated gesture acquired through the motion recognition cameras (1160-2, 1160-3) of FIG. 11B.

[0159] According to one embodiment, at least one display (1150) may provide visual information to a user. For example, at least one display (1150) may include a transparent or translucent lens. At least one display (1150) may include a first display (1150-1) and / or a second display (1150-2) spaced apart from the first display (1150-1). For example, the first display (1150-1) and the second display (1150-2) may be positioned at positions corresponding to the user's left and right eyes, respectively.

[0160] Referring to FIG. 11B, at least one display (1150) can provide a user with visual information transmitted from external light and other visual information distinct from the visual information through a lens included in the at least one display (1150). The lens can be formed based on at least one of a Fresnel lens, a pancake lens, or a multi-channel lens. For example, the at least one display (1150) can include a first surface (1131) and a second surface (1132) opposite the first surface (1131). A display area can be formed on the second surface (1132) of the at least one display (1150). When a user wears the wearable device (1100), external light can be transmitted to the user by being incident on the first surface (1131) and transmitted through the second surface (1132). As another example, at least one display (1150) can display an augmented reality image combined with a virtual reality image provided from at least one optical device (1182, 1184) on a real screen transmitted through external light, in a display area formed on the second surface (1132).

[0161] In one embodiment, at least one display (1150) may include at least one waveguide (1133, 1134) that diffracts light emitted from at least one optical device (1182, 1184) and transmits the diffracted light to a user. The at least one waveguide (1133, 1134) may be formed based on at least one of glass, plastic, or polymer. A nano-pattern may be formed on at least a portion of the exterior or interior of the at least one waveguide (1133, 1134). The nano-pattern may be formed based on a grating structure having a polygonal and / or curved shape. Light incident on one end of the at least one waveguide (1133, 1134) may be propagated to the other end of the at least one waveguide (1133, 1134) by the nano-pattern. At least one waveguide (1133, 1134) may include at least one diffractive element (e.g., a diffractive optical element (DOE), a holographic optical element (HOE)) and at least one reflective element (e.g., a reflective mirror). For example, at least one waveguide (1133, 1134) may be arranged within the wearable device (1100) to guide a screen displayed by at least one display (1150) to the user's eyes. For example, the screen may be transmitted to the user's eyes based on total internal reflection (TIR) ​​occurring within the at least one waveguide (1133, 1134).

[0162] The wearable device (1100) can analyze an object included in a real image collected through a shooting camera (1160-4), combine a virtual object corresponding to an object to be provided with augmented reality among the analyzed objects, and display the virtual object on at least one display (1150). The virtual object can include at least one of text and an image regarding various information related to the object included in the real image. The wearable device (1100) can analyze the object based on a multi-camera such as a stereo camera. For the object analysis, the wearable device (1100) can perform spatial recognition (e.g., simultaneous localization and mapping (SLAM)) using the multi-camera and / or time-of-flight (ToF). A user wearing the wearable device (1100) can view an image displayed on at least one display (1150).

[0163] According to one embodiment, the frame (1100) may be configured as a physical structure that allows the wearable device (1100) to be worn on the user's body. According to one embodiment, the frame (1100) may be configured so that, when the user wears the wearable device (1100), the first display (1150-1) and the second display (1150-2) can be positioned corresponding to the user's left and right eyes. The frame (1100) may support at least one display (1150). For example, the frame (1100) may support the first display (1150-1) and the second display (1150-2) to be positioned corresponding to the user's left and right eyes.

[0164] Referring to FIG. 11A, the frame (1100) may include a region (1120) that at least partially contacts a part of the user's body when the user wears the wearable device (1100). For example, the region (1120) of the frame (1100) that contacts a part of the user's body may include a region that contacts a part of the user's nose, a part of the user's ear, and a part of the side of the user's face that the wearable device (1100) makes contact with. According to one embodiment, the frame (1100) may include a nose pad (1110) that contacts a part of the user's body. When the wearable device (1100) is worn by the user, the nose pad (1110) may contact a part of the user's nose. The frame (1100) may include a first temple (1104) and a second temple (1105) that contact another part of the user's body that is distinct from the part of the user's body.

[0165] For example, the frame (1100) may include a first rim (1101) that surrounds at least a portion of the first display (1150-1), a second rim (1102) that surrounds at least a portion of the second display (1150-2), a bridge (1103) that is disposed between the first rim (1101) and the second rim (1102), a first pad (1111) that is disposed along a portion of the edge of the first rim (1101) from one end of the bridge (1103), a second pad (1112) that is disposed along a portion of the edge of the second rim (1102) from the other end of the bridge (1103), a first temple (1104) that extends from the first rim (1101) and is fixed to a portion of the wearer's ear, and a second temple (1105) that extends from the second rim (1102) and is fixed to a portion of the ear opposite the ear. There are. The first pad (1111) and the second pad (1112) can be in contact with a part of the user's nose, and the first temple (1104) and the second temple (1105) can be in contact with a part of the user's face and a part of the user's ear. The temples (1104, 1105) can be rotatably connected to the rim through the hinge units (1106, 1107) of FIG. 11B. The first temple (1104) can be rotatably connected to the first rim (1101) through the first hinge unit (1106) disposed between the first rim (1101) and the first temple (1104). The second temple (1105) can be rotatably connected to the second rim (1102) via a second hinge unit (1107) disposed between the second rim (1102) and the second temple (1105). In one embodiment, the wearable device (1100) can identify an external object (e.g., a user's fingertip) touching the frame (1100) and / or a gesture performed by the external object by using a touch sensor, a grip sensor, and / or a proximity sensor formed on at least a portion of a surface of the frame (1100).

[0166] According to one embodiment, the wearable device (1100) may include hardwares (e.g., hardwares described above based on the block diagram of FIG. 2) that perform various functions. For example, the hardwares may include a battery module (1170), an antenna module (1175), at least one optical device (1182, 1184), speakers (e.g., speakers 1155-1, 1155-2), a microphone (e.g., microphones 1165-1, 1165-2, 1165-3), a light-emitting module, and / or a printed circuit board (PCB) (1190) (e.g., a printed circuit board). The various hardware components may be arranged within the frame (1100).

[0167] According to one embodiment, microphones (e.g., microphones 1165-1, 1165-2, 1165-3) of the wearable device (1100) may be disposed on at least a portion of the frame (1100) to acquire sound signals. A first microphone (1165-1) disposed on the bridge (1103), a second microphone (1165-2) disposed on the second rim (1102), and a third microphone (1165-3) disposed on the first rim (1101) are illustrated in FIG. 11B , but the number and arrangement of the microphones (1165) are not limited to the embodiment of FIG. 11B . When the number of microphones (1165) included in the wearable device (1100) is two or more, the wearable device (1100) can identify the direction of a sound signal by using a plurality of microphones arranged on different parts of the frame (1100).

[0168] According to one embodiment, at least one optical device (1182, 1184) may project a virtual object onto at least one display (1150) to provide various image information to a user. For example, at least one optical device (1182, 1184) may be a projector. At least one optical device (1182, 1184) may be disposed adjacent to at least one display (1150) or may be included within at least one display (1150) as a part of at least one display (1150). According to one embodiment, the wearable device (1100) may include a first optical device (1182) corresponding to a first display (1150-1) and a second optical device (1184) corresponding to a second display (1150-2). For example, at least one optical device (1182, 1184) may include a first optical device (1182) disposed at an edge of a first display (1150-1) and a second optical device (1184) disposed at an edge of a second display (1150-2). The first optical device (1182) may transmit light to a first waveguide (1133) disposed on the first display (1150-1), and the second optical device (1184) may transmit light to a second waveguide (1134) disposed on the second display (1150-2).

[0169] In one embodiment, the camera (1160) may include a recording camera (1160-4), an eye tracking camera (ET CAM) (1160-1), and / or motion recognition cameras (1160-2, 1160-3). The recording camera (1160-4), the eye tracking camera (1160-1), and the motion recognition cameras (1160-2, 1160-3) may be positioned at different locations on the frame (1100) and may perform different functions. The eye tracking camera (1160-1) may output data indicating the position or gaze of the eyes of a user wearing the wearable device (1100). For example, the wearable device (1100) may detect the gaze from an image including the user's pupils obtained through the eye tracking camera (1160-1).

[0170] The wearable device (1100) can identify an object (e.g., a real object and / or a virtual object) focused on by the user using the user's gaze acquired through the gaze tracking camera (1160-1). The wearable device (1100) that has identified the focused object can execute a function (e.g., gaze interaction) for interaction between the user and the focused object. The wearable device (1100) can express a part corresponding to the eye of an avatar representing the user in a virtual space using the user's gaze acquired through the gaze tracking camera (1160-1). The wearable device (1100) can render an image (or screen) displayed on at least one display (1150) based on the position of the user's eyes.

[0171] For example, the visual quality of a first region related to the gaze within an image and the visual quality (e.g., resolution, brightness, saturation, grayscale, PPI (pixels per inch)) of a second region distinct from the first region may be different from each other. The wearable device (1100) may obtain an image having the visual quality of the first region matching the gaze of the user and the visual quality of the second region using foveated rendering. For example, if the wearable device (1100) supports an iris recognition function, user authentication may be performed based on iris information obtained using the gaze tracking camera (1160-1). Although an example in which the gaze tracking camera (1160-1) is positioned toward the user's right eye is illustrated in FIG. 11B, the embodiment is not limited thereto, and the gaze tracking camera (1160-1) may be positioned solely toward the user's left eye, or toward both eyes.

[0172] In one embodiment, the capturing camera (1160-4) can capture an actual image or background to be aligned with a virtual image to implement augmented reality or mixed reality content. The capturing camera (1160-4) can be used to obtain a high-resolution image based on HR (high resolution) or PV (photo video). The capturing camera (1160-4) can capture an image of a specific object existing at a location viewed by the user and provide the image to at least one display (1150). The at least one display (1150) can display a single image in which information about an actual image or background including an image of the specific object obtained using the capturing camera (1160-4) and a virtual image provided through at least one optical device (1182, 1184) are superimposed. The wearable device (1100) can compensate for depth information (e.g., the distance between the wearable device (1100) and an external object acquired through a depth sensor) using an image acquired through the capture camera (1160-4). The wearable device (1100) can perform object recognition using an image acquired using the capture camera (1160-4). The wearable device (1100) can perform a function of focusing on an object (or subject) in an image (e.g., auto focus) and / or an optical image stabilization (OIS) function (e.g., anti-shake function) using the capture camera (1160-4). The wearable device (1100) can perform a pass-through function to display an image acquired through the capture camera (1160-4) by overlapping at least a portion of a screen representing a virtual space on at least one display (1150) while displaying the screen. In one embodiment, the shooting camera (1160-4) may be positioned on a bridge (1103) disposed between the first rim (1101) and the second rim (1102).

[0173] The gaze tracking camera (1160-1) can implement more realistic augmented reality by tracking the gaze of a user wearing a wearable device (1100) and matching the user's gaze with visual information provided to at least one display (1150). For example, when the wearable device (1100) looks straight ahead, the wearable device (1100) can naturally display environmental information related to the user's front at a location where the user is located on at least one display (1150). The gaze tracking camera (1160-1) can be configured to capture an image of the user's pupil to determine the user's gaze. For example, the gaze tracking camera (1160-1) can receive gaze detection light reflected from the user's pupil and track the user's gaze based on the position and movement of the received gaze detection light. In one embodiment, the gaze tracking camera (1160-1) can be positioned at positions corresponding to the user's left and right eyes. For example, the gaze tracking camera (1160-1) may be positioned within the first rim (1101) and / or the second rim (1102) to face the direction in which the user wearing the wearable device (1100) is positioned.

[0174] The gesture recognition camera (1160-2, 1160-3) can recognize the movement of the user's entire body, such as the user's torso, hand, or face, or a part of the body, and thereby provide a specific event on a screen provided on at least one display (1150). The gesture recognition camera (1160-2, 1160-3) can recognize the user's gesture (gesture recognition), obtain a signal corresponding to the gesture, and provide a display corresponding to the signal on at least one display (1150). The processor can identify the signal corresponding to the gesture, and perform a designated function based on the identification. The gesture recognition camera (1160-2, 1160-3) can be used to perform a spatial recognition function using SLAM and / or a depth map for 6 degrees of freedom pose (6 dof pose). The processor may perform gesture recognition and / or object tracking functions using the motion recognition cameras (1160-2, 1160-3). In one embodiment, the motion recognition cameras (1160-2, 1160-3) may be positioned on the first rim (1101) and / or the second rim (1102).

[0175] The camera (1160) included in the wearable device (1100) is not limited to the above-described gaze tracking camera (1160-1) and motion recognition cameras (1160-2, 1160-3). For example, the wearable device (1100) can identify an external object included in the FoV using a camera positioned toward the user's FoV. The wearable device (1100) can identify an external object based on a sensor for identifying the distance between the wearable device (1100) and the external object, such as a depth sensor and / or a time of flight (ToF) sensor. The camera (1160) positioned toward the FoV can support an autofocus function and / or an optical image stabilization (OIS) function. For example, the wearable device (1100) may include a camera (1160) (e.g., a face tracking (FT) camera) positioned toward the face to obtain an image including the face of a user wearing the wearable device (1100).

[0176] Although not shown, in one embodiment, the wearable device (1100) may further include a light source (e.g., an LED) that emits light toward a subject (e.g., a user's eyes, face, and / or an external object within the FoV) being photographed using the camera (1160). The light source may include an LED having an infrared wavelength. The light source may be disposed on at least one of the frame (1100) and the hinge units (1106, 1107).

[0177] According to one embodiment, the battery module (1170) may supply power to electronic components of the wearable device (1100). In one embodiment, the battery module (1170) may be disposed within the first temple (1104) and / or the second temple (1105). For example, the battery module (1170) may be a plurality of battery modules (1170). The plurality of battery modules (1170) may be disposed within each of the first temple (1104) and the second temple (1105). In one embodiment, the battery module (1170) may be disposed at an end of the first temple (1104) and / or the second temple (1105).

[0178] The antenna module (1175) can transmit signals or power to the outside of the wearable device (1100), or receive signals or power from the outside. In one embodiment, the antenna module (1175) can be positioned within the first temple (1104) and / or the second temple (1105). For example, the antenna module (1175) can be positioned close to one surface of the first temple (1104) and / or the second temple (1105).

[0179] The speaker (1155) can output an audio signal to the outside of the wearable device (1100). The audio output module may be referred to as a speaker. In one embodiment, the speaker (1155) may be positioned within the first temple (1104) and / or the second temple (1105) so as to be positioned adjacent to the ear of a user wearing the wearable device (1100). For example, the speaker (1155) may include a second speaker (1155-2) positioned within the first temple (1104) and thus adjacent to the user's left ear, and a first speaker (1155-1) positioned within the second temple (1105) and thus adjacent to the user's right ear.

[0180] The light-emitting module (not shown) may include at least one light-emitting element. The light-emitting module may emit light of a color corresponding to a specific state or emit light with an action corresponding to a specific state in order to visually provide information regarding a specific state of the wearable device (1100) to the user. For example, when the wearable device (1100) requires charging, it may emit red light at a regular cycle. In one embodiment, the light-emitting module may be disposed on the first rim (1101) and / or the second rim (1102).

[0181] Referring to FIG. 11B, according to one embodiment, a wearable device (1100) may include a printed circuit board (PCB) (1190). The PCB (1190) may be included in at least one of the first temple (1104) or the second temple (1105). The PCB (1190) may include an interposer disposed between at least two sub-PCBs. One or more hardwares included in the wearable device (1100) (e.g., hardwares illustrated by different blocks in FIG. 2) may be disposed on the PCB (1190). The wearable device (1100) may include a flexible PCB (FPCB) for interconnecting the hardwares.

[0182] According to one embodiment, a wearable device (1100) may include at least one of a gyro sensor, a gravity sensor, and / or an acceleration sensor for detecting a posture of the wearable device (1100) and / or a posture of a body part (e.g., a head) of a user wearing the wearable device (1100). Each of the gravity sensor and the acceleration sensor may measure gravitational acceleration and / or acceleration based on mutually perpendicular designated three-dimensional axes (e.g., an x-axis, a y-axis, and a z-axis). The gyro sensor may measure an angular velocity of each of the designated three-dimensional axes (e.g., an x-axis, a y-axis, and a z-axis). At least one of the gravity sensor, the acceleration sensor, and the gyro sensor may be referred to as an inertial measurement unit (IMU). According to one embodiment, the wearable device (1100) may identify a user's motion and / or gesture performed to execute or terminate a specific function of the wearable device (1100) based on the IMU.

[0183] Figures 12a and 12b illustrate an example of an exterior appearance of a wearable device according to one embodiment. The wearable device (1200) of Figures 12a and 12b may include at least a portion of the hardware of the wearable device (1100) described with reference to Figures 11a and / or 11b. An example of an exterior appearance of a first side (1210) of a housing of the wearable device (1200) according to one embodiment is illustrated in Figure 12a, and an example of an exterior appearance of a second side (1220) opposite to the first side (1210) may be illustrated in Figure 12b.

[0184] Referring to FIG. 12A, a first surface (1210) of a wearable device (1200) according to one embodiment may have a form attachable to a body part of a user (e.g., the face of the user). Although not shown, the wearable device (1200) may further include a strap for fixing to a body part of a user, and / or one or more temples (e.g., the first temple (1104) and / or the second temple (1105) of FIGS. 11A and 11B). A first display (1150-1) for outputting an image to a left eye among the user's two eyes, and a second display (1150-2) for outputting an image to a right eye among the two eyes, may be disposed on the first surface (1210). The wearable device (1200) may be formed on the first surface (1210) and may further include a rubber or silicone packing to prevent interference from light (e.g., ambient light) different from the light emitted from the first display (1150-1) and the second display (1150-2).

[0185] According to one embodiment, a wearable device (1200) may include cameras (1160-1) for photographing and / or tracking both eyes of a user adjacent to each of the first display (1150-1) and the second display (1150-2). The cameras (1160-1) may be referred to as the gaze tracking camera (1160-1) of FIG. 11B. According to one embodiment, a wearable device (1200) may include cameras (1160-5, 1160-6) for photographing and / or recognizing a face of a user. The cameras (1160-5, 1160-6) may be referred to as FT cameras. The wearable device (1200) can control an avatar representing the user in a virtual space based on the facial motion of the user identified using cameras (1160-5, 1160-6). For example, the wearable device (1200) can change the texture and / or shape of a portion of the avatar (e.g., a portion of the avatar representing a human face) using information obtained by cameras (1160-5, 1160-6) (e.g., FT cameras) and representing the facial expression of the user wearing the wearable device (1200).

[0186] Referring to FIG. 12B, a camera (e.g., cameras (1160-7, 1160-8, 1160-9, 1160-10, 1160-11, 1160-12)) and / or a sensor (e.g., a depth sensor (1230)) for obtaining information related to the external environment of the wearable device (1200) may be disposed on a second surface (1220) opposite to the first surface (1210) of FIG. 12A. For example, the cameras (1160-7, 1160-8, 1160-9, 1160-10) may be disposed on the second surface (1220) to recognize external objects. The cameras (1160-7, 1160-8, 1160-9, 1160-10) of FIG. 12b can correspond to the motion recognition cameras (1160-2, 1160-3) of FIG. 11b.

[0187] For example, using cameras (1160-11, 1160-12), the wearable device (1200) can obtain images and / or videos to be transmitted to each of the user's eyes. The camera (1160-11) can be placed on the second face (1220) of the wearable device (1200) to obtain an image to be displayed through the second display (1150-2) corresponding to the right eye among the two eyes. The camera (1160-12) can be placed on the second face (1220) of the wearable device (1200) to obtain an image to be displayed through the first display (1150-1) corresponding to the left eye among the two eyes. The cameras (1160-11, 1160-12) can correspond to the shooting camera (1160-4) of FIG. 11B.

[0188] According to one embodiment, the wearable device (1200) may include a depth sensor (1230) disposed on the second face (1220) to identify a distance between the wearable device (1200) and an external object. Using the depth sensor (1230), the wearable device (1200) may obtain spatial information (e.g., a depth map) for at least a portion of the FoV of a user wearing the wearable device (1200). Although not shown, a microphone may be disposed on the second face (1220) of the wearable device (1200) to obtain a sound output from an external object. The number of microphones may be one or more, depending on the embodiment.

[0189] In one embodiment, a method may be required to control execution of a software application based on the location of a wearable device identified within an external space. As described above, according to one embodiment, a wearable device (e.g., wearable device (101) of FIG. 1 , wearable device (1100) of FIG. 11A and / or FIG. 11B , wearable device (1200) of FIG. 12A and / or FIG. 12B ) may include one or more sensors (e.g., sensor (230) of FIG. 2 ), a display (e.g., display (220) of FIG. 2 ), a memory (e.g., memory (215) of FIG. 2 ) for storing instructions, including one or more storage media), and at least one processor (e.g., processor (210) of FIG. 2 ) including a processing circuit. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to define a zone comprising a plurality of regions based on scanning of a physical space. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify a location of the wearable device relative to the zone based on sensor data acquired from the one or more sensors. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to execute a software application associated with the first zone based on identifying that the location of the wearable device is included in a first zone among the plurality of regions included in the zone, and to display a screen provided by the software application on the display to provide a virtual reality experience. In one embodiment, the wearable device may improve a user experience related to virtual reality and / or augmented reality based on the zones that divide the zone.

[0190] For example, the memory may store instructions that, when individually or collectively executed by the at least one processor, cause the wearable device to identify, based on sensor data, that a location of the wearable device has moved out of the first area. The memory may store instructions that, when individually or collectively executed by the at least one processor, cause the wearable device to provide augmented reality based on identifying that the location of the wearable device has moved out of the first area. The memory may store instructions that, when individually or collectively executed by the at least one processor, cause the wearable device to display, while providing augmented reality, a virtual object representing the software application associated with the first area, together with a camera image, based on a field-of-view (FoV) of the wearable device corresponding to the first area.

[0191] For example, the memory may store instructions that, when individually or collectively executed by the at least one processor, cause the wearable device to display another virtual object representing a border line of the first area on the camera image displayed based on the augmented reality.

[0192] For example, the memory may store instructions that, when individually or collectively executed by the at least one processor, cause the wearable device to display the other virtual object in response to detecting, within the camera image, a portion corresponding to a floor included in the first area.

[0193] For example, the memory may store instructions that, when individually or collectively executed by the at least one processor, cause the wearable device to identify whether the wearable device is moved out of the first area using the sensor data while displaying the screen.

[0194] For example, the memory may store instructions that, when individually or collectively executed by the at least one processor, cause the wearable device to adjust a boundary line of the first area on the display based on identifying a location of the wearable device away from the first area while displaying the screen.

[0195] For example, the memory may store instructions that, when individually or collectively executed by the at least one processor, cause the wearable device to display another virtual object (e.g., virtual object (921) of FIG. 9B) on the display to identify whether to stop execution of a software application corresponding to the screen based on identifying a location of the wearable device away from the first area while the wearable device is displaying the screen.

[0196] For example, the memory may store instructions that, when individually or collectively executed by the at least one processor, cause the wearable device to display an icon corresponding to the software application on a portion of the display associated with the first region in response to another input indicating a cessation of execution of the software application.

[0197] For example, the memory may store instructions that, when individually or collectively executed by the at least one processor, cause the wearable device to display, on the display, an image of an external space based on augmented reality and another screen including at least one virtual object included in the virtual reality based on identifying a location of the wearable device apart from the first area while displaying the screen based on virtual reality.

[0198] For example, the memory may store instructions that, when individually or collectively executed by the at least one processor, cause the wearable device to identify one or more software applications associated with the first region based on categories of each of the plurality of software applications stored in the memory of the wearable device and sizes of the plurality of regions.

[0199] According to one embodiment, a method of a wearable device as described above may include an operation of defining a zone including a plurality of areas based on scanning of an actual space. The method may include an operation of identifying a location of the wearable device with respect to the zone based on sensor data acquired from one or more sensors of the wearable device. The method may include an operation of executing a software application related to the first area based on identifying that the location of the wearable device is included in a first area among the plurality of areas included in the zone, and displaying a screen provided by the software application on a display of the wearable device to provide a virtual reality.

[0200] For example, the act of displaying the screen may include an act of identifying, based on sensor data, that the location of the wearable device has moved outside the first area. The act of displaying the screen may include an act of providing augmented reality based on identifying that the location of the wearable device has moved outside the first area. The act of displaying the screen may include an act of displaying, while providing the augmented reality, a virtual object representing the software application associated with the first area, together with a camera image, based on a field-of-view (FoV) of the wearable device corresponding to the first area.

[0201] For example, the action of providing the augmented reality may include an action of displaying another virtual object representing a boundary line of the first area on the camera image displayed based on the augmented reality.

[0202] For example, the method may include, based on identifying a location of the wearable device away from the first area, displaying another virtual object on the display to determine whether to stop execution of a software application corresponding to the screen.

[0203] For example, the act of displaying the other virtual object may include, in response to another input indicating a cessation of execution of the software application, displaying an icon corresponding to the software application on a portion of the display associated with the first region.

[0204] For example, the method may further include an action of displaying, on the display, an image of an external space based on augmented reality and another screen including at least one virtual object included in the virtual reality, based on identifying a location of the wearable device apart from the first area.

[0205] For example, the act of displaying the one or more virtual objects may include an act of identifying one or more software applications associated with the first area based on categories of each of the plurality of software applications stored in the memory of the wearable device and sizes of the plurality of areas.

[0206] As described above, according to one embodiment, a wearable device (e.g., wearable device (101) of FIG. 1 , wearable device (1100) of FIG. 11A and / or FIG. 11B , wearable device (1200) of FIG. 12A and / or FIG. 12B ) may include one or more sensors (e.g., sensor (230) of FIG. 2 ), a display (e.g., display (220) of FIG. 2 ), a memory for storing instructions (e.g., memory (215) of FIG. 2 ), and at least one processor for executing the instructions (e.g., processor (210) of FIG. 2 ). The at least one processor may be configured to, in response to an input related to the execution of a first software application among the plurality of software applications, identify, based on sensor data related to external space obtained from the one or more sensors, a plurality of areas (e.g., areas (140) of FIG. 1 ) corresponding to at least one of the plurality of software applications and a region (e.g., region (130) of FIG. 1 , FIG. 4 , FIG. 5 , FIG. 7a , FIG. 7b , FIG. 9a , FIG. 9b , FIG. 9c , FIG. 9d , FIG. 9e , FIG. 10a and / or FIG. 10b ) including the plurality of areas. The at least one processor may be configured to display, on the display, a first screen provided by the first software application based on identifying a location of the wearable device included in a first area among the plurality of areas corresponding to the first software application. The at least one processor may be configured to display a second screen for guiding movement toward the first area based on identifying a location of the wearable device away from the first area.

[0207] For example, the at least one processor may be configured to display the second screen including a virtual object superimposed on a portion of the display associated with the first region.

[0208] For example, the at least one processor may be configured to display the virtual object having a shape representing a boundary line of the first area.

[0209] For example, the at least one processor may be configured to display a virtual object for receiving another input to stop execution of the first software application based on identifying the location of the wearable device away from the first area while displaying the first screen.

[0210] According to one embodiment of the present invention, a method of a wearable device as described above may include, in response to an input related to the execution of a first software application among a plurality of software applications, an operation of identifying a plurality of areas corresponding to at least one of the plurality of software applications and a zone including the plurality of areas based on sensor data related to external space obtained from one or more sensors of the wearable device. The method may include an operation of displaying a first screen provided by the first software application on a display of the wearable device based on identifying a location of the wearable device included in a first area corresponding to the first software application among the plurality of areas. The method may include an operation of displaying a second screen for guiding movement to the first area based on identifying a location of the wearable device spaced apart from the first area.

[0211] For example, the act of displaying the second screen may include an act of displaying the second screen including a virtual object superimposed on a portion of the display associated with the first area.

[0212] For example, the action of displaying the second screen may include an action of displaying the virtual object having a form representing a boundary line of the first area.

[0213] For example, the act of displaying the first screen may include an act of displaying a virtual object for receiving another input for stopping execution of the first software application based on identifying the location of the wearable device away from the first area while displaying the first screen.

[0214] As described above, according to one embodiment, a wearable device (e.g., wearable device (101) of FIG. 1 , wearable device (1100) of FIG. 11A and / or FIG. 11B , wearable device (1200) of FIG. 12A and / or FIG. 12B ) may include one or more sensors (e.g., sensor (230) of FIG. 2 ), a display (e.g., display (220) of FIG. 2 ), a memory for storing instructions (e.g., memory (215) of FIG. 2 ), and at least one processor for executing the instructions (e.g., processor (210) of FIG. 2 ). The at least one processor may be configured to obtain sensor data related to a location of the wearable device from the one or more sensors while displaying a first screen related to virtual reality or augmented reality. The at least one processor may be configured to display, on the display, a virtual object for adjusting a size of the first region within a zone (e.g., zone (130) of FIGS. 1, 4, 5, 7a, 7b, 9a, 9b, 9c, 9d, 9e, 10a and / or 10b) that includes a plurality of zones (e.g., zones (140) of FIG. 1) that are spaced apart from a first region corresponding to the first screen and include the first region, based on identifying the location of the wearable device using the sensor data. The at least one processor may be configured to stop displaying the first screen on the display based on identifying the location of the wearable device spaced apart from the zone using the sensor data.

[0215] For example, the at least one processor may be configured to display another virtual object on the display to stop execution of a software application corresponding to the first screen based on identifying the location of the wearable device, which is located within the zone and is spaced apart from the first area.

[0216] For example, the at least one processor may be configured to display another virtual object representing a software application corresponding to the first screen on a portion of the display associated with the first area, based on identifying the location of the wearable device away from the area.

[0217] According to one embodiment, a method of a wearable device as described above may include an operation of acquiring sensor data related to a location of the wearable device from one or more sensors of the wearable device while displaying a first screen related to virtual reality or augmented reality. The method may include an operation of displaying a virtual object on a display of the wearable device for adjusting a size of the first area within the area based on identifying a location of the wearable device within a zone that includes a plurality of areas that are spaced apart from a first area corresponding to the first screen and that include the first area using the sensor data. The method may include an operation of ceasing to display the first screen on the display based on identifying a location of the wearable device that is spaced apart from the area using the sensor data.

[0218] For example, the action of displaying the virtual object may include the action of displaying another virtual object to stop execution of a software application corresponding to the first screen.

[0219] For example, the interrupting action may include displaying another virtual object representing a software application corresponding to the first screen on a portion of the display linked to the first area.

[0220] The devices described above may be implemented as hardware components, software components, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments may be implemented using one or more general-purpose computers or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding to them. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device is sometimes described as being used alone; however, one of ordinary skill in the art will recognize that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, a processing unit may include multiple processors, or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.

[0221] Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing device to perform a desired operation or may independently or collectively command the processing device. The software and / or data may be embodied in any type of machine, component, physical device, computer storage medium, or device for interpretation by the processing device or for providing instructions or data to the processing device. The software may also be distributed over networked computer systems and stored or executed in a distributed manner. The software and data may be stored on one or more computer-readable recording media.

[0222] The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. In this case, the medium may be one that continuously stores a computer-executable program or one that temporarily stores it for execution or download. In addition, the medium may be various recording or storage means in the form of a single or multiple hardware combinations, and is not limited to a medium directly connected to a computer system, but may also be distributed over a network. Examples of the medium may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and those configured to store program commands, including ROM, RAM, and flash memory. In addition, examples of other media may include recording or storage media managed by app stores that distribute applications, sites that supply or distribute various software, servers, etc.

[0223] Although the embodiments described above have been described by way of limited examples and drawings, those skilled in the art will appreciate that various modifications and variations can be made based on the above teachings. For example, appropriate results can still be achieved even if the described techniques are performed in a different order than described, and / or components of the described systems, structures, devices, circuits, etc. are combined or combined in a different manner than described, or are replaced or substituted with other components or equivalents.

[0224] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims described below.

Claims

1. In a wearable device (101; 1100; 1200), One or more sensors (230); display (220); A memory (215) storing instructions, including one or more storage media; and At least one processor including a processing circuit, wherein the instructions, when individually or collectively executed by the at least one processor, cause the wearable device to: Define a region containing multiple areas based on scanning of the physical space; Identifying the location of the wearable device relative to the area based on sensor data acquired from the one or more sensors; and Causing the wearable device to execute a software application related to the first area based on identifying that the location of the wearable device is included in a first area among the plurality of areas included in the zone, thereby displaying a screen provided by the software application on the display to provide a virtual reality. Wearable devices.

2. In claim 1, the instructions, when individually or collectively executed by the at least one processor, cause the wearable device to: Based on the sensor data, identifying that the location of the wearable device has moved out of the first area; Providing augmented reality based on identifying that the position of the wearable device has moved out of the first area; and While providing augmented reality, storing the instructions that cause a virtual object representing the software application related to the first area to be displayed, together with a camera image, based on a field-of-view (FoV) of the wearable device corresponding to the first area. Wearable devices.

3. In claim 2, the instructions, when individually or collectively executed by the at least one processor, cause the wearable device to: Storing the instructions, which cause another virtual object representing a border line of the first area to be displayed on the camera image displayed based on the augmented reality, Wearable devices.

4. In claim 3, the instructions, when individually or collectively executed by the at least one processor, cause the wearable device to: storing the instructions, which cause the display of the other virtual object in response to detecting a portion corresponding to the floor included in the first area within the camera image; Wearable devices.

5. In claims 1 to 4, the instructions, when individually or collectively executed by the at least one processor, cause the wearable device to: storing the instructions, which cause the boundary line of the first area to be adjusted based on identifying the position of the wearable device away from the first area while displaying the screen; Wearable devices.

6. In claims 1 to 5, the instructions, when individually or collectively executed by the at least one processor, cause the wearable device to: storing the instructions, which cause another virtual object (921) to be displayed on the display to identify whether to stop execution of a software application corresponding to the screen, based on identifying a location of the wearable device apart from the first area while displaying the screen; Wearable devices.

7. In claims 1 to 6, the instructions, when individually or collectively executed by the at least one processor, cause the wearable device to: storing said instructions, in response to another input indicating the termination of execution of said software application, causing an icon corresponding to said software application to be displayed on a portion of said display associated with said first region; Wearable devices.

8. In claims 1 to 7, the instructions, when individually or collectively executed by the at least one processor, cause the wearable device to: storing instructions that cause one or more software applications associated with the first area to be identified based on the categories of each of the plurality of software applications stored in the memory of the wearable device and the sizes of the plurality of areas; Wearable devices.

9. In the method of a wearable device, An action of defining a zone containing multiple areas based on scanning of a real space; An action of identifying a location of the wearable device with respect to the area based on sensor data obtained from one or more sensors of the wearable device; An operation of displaying a screen provided by the software application on a display of the wearable device to provide virtual reality, based on identifying that the location of the wearable device is included in a first area among the plurality of areas included in the zone, by executing a software application related to the first area. method.

10. In claim 9, the action of displaying the screen is: An action of identifying, based on sensor data, that the location of the wearable device has moved out of the first area; An operation for providing augmented reality based on identifying that the position of the wearable device has moved out of the first area; and While providing the augmented reality, an action is included to display a virtual object representing the software application related to the first area, together with a camera image, based on a field-of-view (FoV) of the wearable device corresponding to the first area. method.

11. In claim 10, the operation of providing the augmented reality comprises: Including an action of displaying another virtual object representing a boundary line of the first area on the camera image displayed based on the augmented reality. method.

12. In claims 9 to 11, Further comprising an action of displaying another virtual object on the display to identify whether to stop execution of a software application corresponding to the screen based on identifying a location of the wearable device spaced from the first area. method.

13. In claim 12, the action of displaying the other virtual object is: In response to another input indicating the termination of execution of said software application, further comprising the action of displaying an icon corresponding to said software application on a portion of said display associated with said first region. method.

14. In claims 9 to 13, the operation of displaying one or more virtual objects comprises: An operation of identifying one or more software applications associated with the first area based on categories of each of the plurality of software applications stored in the memory of the wearable device and sizes of the plurality of areas, method.

15. A non-transitory computer-readable storage medium storing instructions, wherein the instructions, when executed by a wearable device, cause the wearable device to perform the operations of claims 9 to 14. A non-transitory computer-readable storage medium.

Images