Wearable device and method for outputting notification about reference boundary line formed in external space to provide virtual space

Abstract

This wearable device may: configure a safety zone for a space of an external environment for a user's safety; provide a virtual space by using displays configured to be arranged toward the user's eyes; at a first position within the safety zone, while the virtual space is being provided beyond the safety zone, provide both a video of the external environment, which is being acquired using a camera, and the virtual space by using the displays in order to visually emphasize the external environment with respect to the virtual space, on the basis of identifying the wearable device that is moving out of the safety zone; and at a second position within the safety zone, while the virtual space being is provided, stop providing of the virtual space on the basis of identifying the wearable device that is moving out of the safety zone.

Description

Wearable device and method for outputting a notification regarding a reference boundary line formed in an external space to provide a virtual space

[0001] The present disclosure relates to a wearable device and method for outputting a notification regarding a reference boundary line formed in an external space to provide a virtual space.

[0002] To provide an enhanced user experience, electronic devices are being developed that provide augmented reality (AR) services, which display computer-generated information in conjunction with external objects in the real world. These electronic devices may be wearable devices worn by the user. For example, these electronic devices may be AR glasses and / or head-mounted devices (HMDs).

[0003] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art related to the present disclosure.

[0004] According to an embodiment, a wearable device may include displays configured to be positioned toward the eyes of a user wearing the wearable device, at least one camera configured to be positioned toward an external environment, at least one sensor configured to obtain information about the external environment, a memory including one or more storage media for storing instructions, and at least one processor including a processing circuit. The instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to set a safety zone with respect to a space in the external environment for the safety of the user wearing the wearable device. The instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to provide a virtual space using the displays. The instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to provide, using the displays, a video of the external environment being acquired using the at least one camera and the virtual space together to visually emphasize the external environment relative to the virtual space, based on identifying the wearable device moving out of the safety zone while the virtual space is provided beyond the safety zone at a first location of the wearable device within the safety zone.The instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to stop providing the virtual space based on identifying the wearable device moving out of the safety zone while the virtual space is being provided at a second location of the wearable device within the safety zone.

[0005] In one embodiment, a non-transitory computer-readable storage medium storing instructions may be provided. The instructions, when executed by a wearable device, including displays configured to be positioned toward a user's eyes, at least one camera configured to be positioned toward an external environment, and at least one sensor configured to obtain information about the external environment, may cause the wearable device to set a safety zone in a space of the external environment for the safety of a user wearing the wearable device. The instructions, when executed by the wearable device, may cause the wearable device to provide a virtual space using the displays. The instructions, when executed by the wearable device, may cause the wearable device to provide, using the displays, a video of the external environment being acquired using the at least one camera and the virtual space together, by increasing the transparency of the virtual space, based on identifying the wearable device moving out of the safety zone while the virtual space is provided at a first location of the wearable device within the safety zone. The instructions, when executed by the wearable device, may cause the wearable device to stop providing the virtual space, based on identifying the wearable device moving out of the safety zone while the virtual space is provided at a second location of the wearable device within the safety zone.

[0006] In one embodiment, a method of a wearable device may be provided. The wearable device may include displays configured to be positioned toward the eyes of a user wearing the wearable device, at least one camera configured to be positioned toward an external environment, and at least one sensor configured to acquire information about the external environment. The method may include providing a first portion of the virtual space using the displays while the wearable device is positioned within a safety zone set for a space of the external environment for the safety of a user wearing the wearable device. The method may include stopping providing the virtual space using the displays and providing a video of the external environment, which is being acquired using the at least one camera, using the displays based on the wearable device being moved out of the safety zone through a first boundary that is a portion of the boundary of the safety zone. The method may include an operation of identifying a second portion of the virtual space based on a location of the wearable device outside the safety zone, based on the wearable device moving out of the safety zone through a second boundary that is part of the boundary of the safety zone and different from the first boundary. The method may include an operation of providing a video of the external environment and the second portion of the virtual space using the displays, based on the identification of the second portion of the virtual space.

[0007] In one embodiment, a wearable device may include displays configured to be positioned toward the eyes of a user wearing the wearable device, at least one camera configured to be positioned toward an external environment, at least one sensor configured to obtain information about the external environment, a memory including one or more storage media for storing instructions, and at least one processor including a processing circuit. The instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to provide a first portion of the virtual space using the displays while the wearable device is positioned within a safety zone set for a space of the external environment for the safety of a user wearing the wearable device. The instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to stop providing the virtual space using the displays and to provide, using the displays, a video of the external environment being acquired using the at least one camera, based on the wearable device moving out of the safety zone through a first boundary that is part of the boundaries of the safety zone. The instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to identify, based on a location of the wearable device outside the safety zone, a second portion of the virtual space, based on the wearable device moving out of the safety zone through a second boundary that is part of the boundaries of the safety zone and is different from the first boundary.The instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to provide, using the displays, a video of the external environment and the second portion of the virtual space based on identifying the second portion of the virtual space.

[0008] In one embodiment, a non-transitory computer-readable storage medium storing instructions may be provided. The instructions may be executed by a wearable device, the wearable device including displays configured to be positioned toward a user's eyes, at least one camera configured to be positioned toward an external environment, and at least one sensor configured to obtain information about the external environment. The instructions, when executed by the wearable device, may cause the wearable device to provide a virtual space using the displays while the wearable device is positioned within a safety zone set for the safety of a user wearing the wearable device. The instructions, when executed by the wearable device, may cause the wearable device to display a virtual object for changing at least one of a placement position or a placement direction of the virtual space relative to the safety zone based on identifying, using the at least one camera and the at least one sensor, that the wearable device has moved toward the boundary of the safety zone. The instructions, when executed by the wearable device, may cause the wearable device to, in response to an input related to the virtual object, change at least one of the placement position or the placement direction of the virtual space, and identify a portion of the virtual space based on a position and orientation of the wearable device in the virtual space resulting from the change. The instructions, when executed by the wearable device, may cause the wearable device to provide, using the displays, the identified portion of the virtual space.

[0009] In one embodiment, a wearable device may include displays configured to be positioned toward the eyes of a user wearing the wearable device, at least one camera configured to be positioned toward an external environment, at least one sensor configured to obtain information about the external environment, a memory including one or more storage media for storing instructions, and at least one processor including a processing circuit. The instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to provide a virtual space using the displays while the wearable device is positioned within a safety zone set for the safety of a user wearing the wearable device. The instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to display a virtual object for changing at least one of a placement position or a placement orientation of the virtual space relative to the safety zone based on identifying, using the at least one camera and the at least one sensor, that the wearable device has moved toward the boundary of the safety zone. The instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to, in response to an input related to the virtual object, change at least one of the placement position or the placement orientation of the virtual space, and identify a portion of the virtual space based on a position and orientation of the wearable device in the virtual space resulting from the change. The instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to provide, using the displays, the portion of the identified virtual space.

[0010] FIG. 1 illustrates exemplary screens displayed by a wearable device according to one embodiment and based on a geographic location.

[0011] FIG. 2 illustrates a block diagram of a wearable device according to one embodiment.

[0012] FIG. 3 illustrates a flowchart of a wearable device according to one embodiment.

[0013] FIG. 4 illustrates an exemplary operation of a wearable device receiving an input for setting a safety zone configured to provide a virtual space.

[0014] Figure 5 illustrates an exemplary flowchart of the operation of a wearable device providing a virtual space.

[0015] FIG. 6 illustrates an exemplary reference area formed to determine whether a wearable device is approaching the boundary of a safety zone.

[0016] FIGS. 7a, 7b, 7c, and 7d illustrate exemplary screens displayed by a wearable device to indicate the boundaries of a safety zone.

[0017] FIG. 8 illustrates a flowchart of a wearable device according to one embodiment.

[0018] FIG. 9A, FIG. 9B, and FIG. 9C illustrate exemplary operations of a wearable device for detecting movement of a user wearing the wearable device.

[0019] FIG. 10A and FIG. 10B illustrate exemplary screens displayed by a wearable device according to geographic location.

[0020] FIG. 11 illustrates a flowchart of a wearable device according to one embodiment.

[0021] Figures 12a, 12b, and 12c illustrate exemplary operations of a wearable device that change the relationship between the wearable device's geographic location and virtual space.

[0022] Figures 13a, 13b, and 13c illustrate exemplary operations of a wearable device that change the relationship between the wearable device's geographic location and a virtual object.

[0023] FIG. 14A illustrates an example of a perspective view of a wearable device according to one embodiment.

[0024] FIG. 14b illustrates an example of one or more hardware elements disposed within a wearable device, according to one embodiment.

[0025] FIGS. 15A and 15B illustrate an example of an appearance of a wearable device according to one embodiment.

[0026] Hereinafter, various embodiments of this document are described with reference to the attached drawings.

[0027] The various embodiments of this document and the terminology used therein are not intended to limit the technology described in this document to a specific embodiment, but should be understood to include various modifications, equivalents, and / or substitutes of the embodiment. In connection with the description of the drawings, similar reference numerals may be used for similar components. The singular expression may include plural expressions unless the context clearly indicates otherwise. In this document, expressions such as "A or B", "at least one of A and / or B", "A, B, or C", or "at least one of A, B, and / or C" may include all possible combinations of the items listed together. Expressions such as "first", "second", "first", or "second" may modify the corresponding components regardless of order or importance, and are only used to distinguish one component from another, but do not limit the corresponding components. When it is said that a component (e.g., a first component) is “(functionally or communicatively) connected” or “connected” to another component (e.g., a second component), said component may be directly connected to said other component, or may be connected via another component (e.g., a third component).

[0028] The term "module" as used in this document includes a unit composed of hardware, and may be used interchangeably with terms such as "component" or "circuit." A module may be an integrally composed component, or a minimal unit or portion thereof that performs one or more functions. For example, a module may be composed of an application-specific integrated circuit (ASIC).

[0029] FIG. 1 illustrates exemplary screens (181, 182, 183, 184) displayed by a wearable device (101) according to one embodiment and based on a geographic location. The wearable device (101) may include a head-mounted display (HMD) that can be worn on the head of a user (110). The wearable device (101) may be referred to as a head-mounted display (HMD) device, a headgear electronic device, an eyeglass-type (or goggle-type) electronic device, a video see-through (VST) device, an extended reality (XR) device, a virtual reality (VR) device, and / or an augmented reality (AR) device. Although the external appearance of the wearable device (101) having the form of eyeglasses is illustrated, the embodiment is not limited thereto. An example of a hardware configuration included in a wearable device (101) is exemplarily described with reference to FIG. 2. An example of a structure of a wearable device (101) that can be worn on the head of a user (110) is described with reference to FIGS. 14a, 14b, 15a, and / or 15b. The wearable device (101) may be referred to as an electronic device. For example, the electronic device may include an accessory (e.g., a strap) for attaching to the head of a user (110).

[0030] According to one embodiment, a wearable device (101) may perform functions related to augmented reality (AR) and / or mixed reality (MR). For example, when a user (110) wears the wearable device (101), the wearable device (101) may include at least one lens positioned adjacent to the user's (110) eyes. The wearable device (101) may combine ambient light passing through the lens with light emitted from a display of the wearable device (101). A display area of ​​the display may be formed within the lens through which the ambient light passes. Since the wearable device (101) combines the ambient light and the light emitted from the display, the user (110) may see a mixed image of a real object recognized by the ambient light and a virtual object formed by the light emitted from the display. The augmented reality, mixed reality, and / or virtual reality described above may be referred to as extended reality (XR).

[0031] In one embodiment, the wearable device (101) may perform functions related to video see-through (or visible see-through), pass-through, and / or virtual reality (VR). For example, when a user (110) wears the wearable device (101), the wearable device (101) may include a housing that covers both eyes of the user (110). The wearable device (101) may include displays configured to be positioned toward the eyes of the user (110) wearing the wearable device (101) (e.g., the displays described with reference to FIGS. 14A, 14B, 15A, and / or 15B). The wearable device (101) may include cameras configured to be positioned toward the external environment. The wearable device (101) can display images and / or videos captured through cameras on the displays. For example, images and / or videos of the external environment can be displayed on the displays to provide a user experience similar to that of directly viewing the external environment to the user (110).

[0032] Referring to FIG. 1, an embodiment of a wearable device (101) worn by a user (110) is illustrated. Within the exemplary state of FIG. 1, the wearable device (101) may provide a virtual space to the user (110) wearing the wearable device (101) by using displays (e.g., displays positioned to face the two eyes of the user (110) wearing the wearable device (101). For example, in order to provide a VR-based user experience, the wearable device (101) may display a screen (e.g., an image and / or video) representing at least a portion of the virtual space on the entire display area of ​​the displays covering the two eyes of the user (110).

[0033] Referring to FIG. 1, exemplary screens (181, 182, 183, 184) displayed on at least one display are illustrated to provide a VR-based user experience. For example, the screens (181, 182, 183, 184) may be displayed on at least one of the displays configured to be positioned toward the two eyes of the user (110). For example, while the wearable device (101) is positioned at a location (or geographical location) (p1) in the external environment, the wearable device (101) may display the screen (181) on at least one display. While providing a VR-based virtual space, the wearable device (101) may visualize, provide, or display a view of a portion of the virtual space through the screen (181). The wearable device (101) can provide a user experience that is separated (or disconnected) from the external environment to a user (110) wearing the wearable device (101) by using a screen (181) that expresses a virtual space that is completely different from the external environment.

[0034] In one embodiment, the display of images and / or videos (e.g., images and / or videos of a virtual space) rendered independently of the external environment by the wearable device (101), such as screen (181), may be conditionally performed within a safety zone (130) set for the safety of a user (110) wearing the wearable device (101). The safety zone (130) may be referred to as a play area, a protection area, a protection area, a geographical area, a safety area, a safety zone, a guardian area, a guardian zone, and / or a guardian. The boundary (or virtual boundary) of the safety zone (130) may be referred to as a fence and / or a wall in terms of a boundary separated from the external environment. The operation of the wearable device (101) for setting and / or creating the boundary of the safety zone (130) is described with reference to FIG. 3 and / or FIG. 4.

[0035] Referring to FIG. 1, the wearable device (101) may provide a first part of a virtual space linked to the safety zone (130) using displays while the wearable device (101) is positioned within a safety zone (130) set for the safety of a user wearing the wearable device (101). When the virtual space is linked to the safety zone (130), the wearable device (101) may change the view of the virtual space displayed on the displays according to the geographic location of the wearable device (101) (e.g., the location inside the safety zone (130)). For example, the wearable device (101) may change the view of the virtual space according to the movement of the wearable device (101), thereby providing a user experience similar to that of the user (110) moving within the virtual space. For example, at a location (p1) inside the safety zone (130), the wearable device (101) can display a screen (181) representing a view of the virtual space as seen from a location within the virtual space mapped to the location (p1). When a user (110) wearing the wearable device (101) moves, the wearable device (101) can detect a change in the location of the wearable device (101).

[0036] For example, when a user (110) wearing a wearable device (101) moves from a location (p1) to a location (p2), the wearable device (101) may display a screen (182) representing a view of the virtual space as seen from a location within the virtual space linked to the location (p2). Similarly, when a user (110) wearing a wearable device (101) moves from a location (p1) to a location (p4), the wearable device (101) may display a screen (184) representing a view of the virtual space as seen from a location within the virtual space corresponding to the location (p4). Referring to the screens (182, 184), the virtual space may be defined or provided beyond the safety zone (130). Referring to FIG. 1, while positioned within a safety zone (130), such as at locations (p1, p2, p4), the wearable device (101) can display a screen (e.g., screens (181, 182, 184)) representing at least a portion of a virtual space. While positioned within the safety zone (130), the wearable device (101) can provide a virtual environment among an external environment and a VR-based virtual environment.

[0037] A safety zone (130) may be set or defined to transition between a virtual environment and an external environment. For example, when moving from the inside to the outside of the safety zone (130), the wearable device (101) may at least temporarily (or at least partially) stop providing the virtual environment using the displays. Referring to FIG. 1 , to prevent collisions between external objects (e.g., a sofa (121), a lamp (122), and / or a bookshelf (123)) and the user (110), the safety zone (130) may be formed in an empty space separated from the external objects.

[0038] Referring to exemplary screens (181, 183, 184) displayed by a wearable device (101) positioned inside a safety zone (130), a virtual space can be formed independently of the external environment (and / or the safety zone (130)). For example, the virtual space expressed through the screen can be expressed according to dimensions that are different from the dimensions of the safety zone (130) (e.g., width, length, height, area, and / or volume). Referring to FIG. 1, a screen (184) displayed at a position (p4) adjacent to a bookshelf (123) can include a view of the virtual space that represents a movable empty space extending forward, even though a user (110) wearing the wearable device (101) is standing facing the bookshelf (123) at a position (p4) adjacent to the bookshelf (123). For example, a user (110) viewing the screen (184) can continue to move toward the bookshelf (123) to move toward an empty space in the virtual space.

[0039] According to one embodiment, the wearable device (101) may execute a function to notify that the user (110) viewing the screen (184) has moved outside the safety zone (130) in order to prevent the user (110) from colliding with an external object (e.g., a bookshelf (123)) outside the safety zone (130) according to the view of the virtual space included in the screen (184). The function may be related to the possibility of collision by an external object (e.g., a sofa (121), a lamp (122), and / or a bookshelf (123)) located outside the safety zone (130). For example, the wearable device (101) may display a virtual object visualizing the possibility of collision based on the position of the wearable device (101) approaching the boundary of the safety zone (130). For example, the wearable device (101) may output the possibility of a collision through a haptic actuator configured to output vibration and / or a speaker configured to output sound. For example, the wearable device (101) may display a video being acquired through a camera to provide the user (110) with a view of the adjacent external environment together with a view of the virtual space. An exemplary operation of the wearable device (101) that executes a function to notify of movement outside the safety zone (130) is described with reference to FIGS. 5, 6, and 7A to 7D.

[0040] In one embodiment, the boundary of the safety zone (130) may include a first boundary configured to stop providing virtual space. Based on detecting that the wearable device (101) has moved outside the safety zone (130) through the first boundary, the wearable device (101) may refrain from providing at least a portion of the virtual space using the displays. Based on detecting that the wearable device (101) has moved outside the safety zone (130) through the first boundary, the wearable device (101) may provide video being acquired using at least one camera using the displays. For example, the wearable device (101) may switch to pass-through mode (or VST mode) to display images and / or video of the external environment. When the location of the wearable device (101) moving outside the safety zone (130) through the first boundary is identified, the wearable device (101) can switch from a mode that provides a virtual space through the entire display area of ​​the displays (e.g., VR mode) to a pass-through mode.

[0041] In one embodiment, the boundary of the safety zone (130) may include a second boundary that is set to conditionally maintain provision of a virtual space. The second boundary may be a remaining boundary of the safety zone (130) that does not overlap with the first boundary and is different from the first boundary. Based on detecting the wearable device (101) moving outside the safety zone (130) through the second boundary, the wearable device (101) may identify a second portion of the virtual space connected to the first portion of the virtual space associated with the safety zone (130). Referring to FIG. 1 , when the wearable device (101) is located at a location (p3) outside the safety zone (130), the wearable device (101) may identify a second portion of the virtual space associated with the location (p3) of the wearable device (101) (e.g., a portion associated with the location (p3)). Based on identifying the second part, the wearable device (101) can provide the second part using displays. Together with the second part, the wearable device (101) can provide a mixed video (e.g., a video acquired through at least one camera) using a specified transparency.

[0042] Referring to FIG. 1, an exemplary screen (183) displayed by a wearable device (101) positioned at a location (p3) outside a safety zone (130) is illustrated. The screen (183) may include a view of a virtual space as seen from a location within the virtual space corresponding to the location (p3). Within the screen (183), images and / or videos representing the virtual space may be mixed with images and / or videos representing the external environment, depending on a specified transparency. For example, the wearable device (101) may display, on the displays, a first video of the virtual space based on a first transparency, and a second video of a camera based on a second transparency. As a non-limiting example, the first video may be superimposed on the second video.

[0043] Referring to an exemplary screen (183) of FIG. 1, since the wearable device (101) located at location (p3) provides a video of the external environment together with the virtual space, a user (110) wearing the wearable device (101) can simultaneously view the virtual space and the external environment. For example, the wearable device (101) can display a video representing an external object located in front of the wearable device (101), such as a dog (124), on the displays. For example, a visual object (140) representing a dog (124) included in the video can be included in the screen (183). Together with the visual object (140) representing the external object of the external environment, the wearable device (101) can provide a view of the virtual space as seen from a location in the virtual space linked to location (p3).

[0044] For example, when moving from a safety zone (130) to a movable external environment, the wearable device (101) may, instead of ceasing to provide a virtual space, provide images and / or videos of the external environment together with the virtual space, thereby maintaining a user experience based on the virtual space. While maintaining a user experience based on the virtual space, the wearable device (101) may provide information about the external environment to the user (110) using images and / or videos of the external environment. The operation of the wearable device (101) to maintain a user experience based on the virtual space in an external environment is described with reference to FIGS. 8, 9A, 9B, 9C, 10A, and / or 10B.

[0045] In one embodiment, the boundary of the safety zone (130) adjacent to the bookshelf (123) may be a first boundary set to stop providing the virtual space. At a location (p4) adjacent to the first boundary, the wearable device (101) may display a virtual object (150) for changing properties of the virtual space (e.g., the placement location, placement direction, coordinate system, direction, and / or relationship with the safety zone (130) of the virtual space) together with a notification related to the first boundary. For example, while displaying a screen (184) representing a movable empty space extending forward, based on receiving an input related to the virtual object (150), the wearable device (101) may change the properties of the virtual space such that the empty space extends in a direction different from the direction toward the bookshelf (123). The wearable device (101) may use the displays to provide the virtual space based on the changed properties. The operation of a wearable device (101) based on input related to a virtual object (150) is described with reference to FIG. 11, FIG. 12a, FIG. 12b, FIG. 12c, FIG. 13a, FIG. 13b, and / or FIG. 13c.

[0046] As described above, according to one embodiment, the wearable device (101) can provide a virtual space for VR while positioned inside the safety zone (130). When moving toward the boundary of the safety zone (130), the wearable device (101) can execute a function for notifying that it is moving toward the boundary. The function can be differentially executed depending on the distance between the boundary and an external object. The wearable device (101) can classify the boundary of the safety zone (130) into a first boundary for directly stopping providing the virtual space and a second boundary for conditionally maintaining providing the virtual space. When the wearable device (101) moves outside the safety zone (130) through the first boundary, the wearable device (101) can stop providing the virtual space. When the wearable device (101) moves outside the safety zone (130) through the second boundary, the wearable device (101) can display a video of the external environment as an overlay on the virtual space, such as the screen (183).

[0047] Below, with reference to FIG. 2, an exemplary hardware configuration of the wearable device (101) of FIG. 1 is described.

[0048] FIG. 2 illustrates a block diagram of a wearable device (101) according to one embodiment. The wearable device (101) of FIG. 2 may include the wearable device (101) of FIG. 1.

[0049] Referring to FIG. 2, a wearable device (101) according to one embodiment may include a processor (210), a memory (215), a display (220), a camera (225) (e.g., an eye tracking camera (225-1), an external camera (225-2)), a sensor (230), a communication circuit (235), or any combination thereof. The processor (210), the memory (215), the display (220), the camera (225), the sensor (230), and the communication circuit (235) may be electrically and / or operatively connected to each other by electronic components such as a communication bus (202). In the present disclosure, the operative connection of the electronic components may include a direct connection established between the electronic components and / or an indirect connection established between the electronic components such that a first electronic component among the electronic components is controlled by a second electronic component among the electronic components. The type and / or number of electronic components included in the wearable device (101) is not limited to those illustrated in FIG. 2. For example, the wearable device (101) may include only some of the electronic components illustrated in FIG. 2.

[0050] According to one embodiment, a processor (210) of a wearable device (101) may include a circuit (e.g., a processing circuit) for processing data based on instructions. The processor (210) may include a central processing unit (CPU), a graphic processing unit (GPU), and / or a neural processing unit (NPU). The processor (210) may be referred to as an application processor (AP). The wearable device (101) including the processor (210) may include cases where the wearable device (101) includes at least one processor, the wearable device (101) includes a plurality of processors, and / or the wearable device (101) includes a single processor. The processor (210) may have a multi-core processor structure such as a dual core, a quad core, a hexa core, and / or an octa core. The multi-core processor architecture of the processor (210) may include a structure (e.g., a big-little architecture) having different core circuits that are distinguished by power consumption, clock, and / or calculation amount per unit time. At least one processor included in the wearable device (101) may individually and / or collectively perform the operations and / or functions of the present disclosure.

[0051] According to one embodiment, the memory (215) of the wearable device (101) may include an electronic component (or storage medium) for storing data and / or instructions input to and / or output from the processor (210). The memory (215) may include, for example, a volatile memory such as a random-access memory (RAM) and / or a non-volatile memory such as a read-only memory (ROM). The volatile memory may include, for example, at least one of a dynamic RAM (DRAM), a static RAM (SRAM), a cache RAM, and a pseudo SRAM (PSRAM). The non-volatile memory may include, for example, at least one of a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), a flash memory, a hard disk, a compact disc, and an embedded multi media card (eMMC). In one embodiment, memory (215) may be referred to as storage.

[0052] In one embodiment, the display (220) of the wearable device (101) may be configured to visualize information (or signals) provided from the processor (210). The display (220) may include a flexible display, a flat panel display (FPD), and / or electronic paper. The display (220) may include a liquid crystal display (LCD), a plasma display panel (PDP), and / or a plurality of light emitting diodes (LEDs). The LEDs of the display (220) may be implemented as organic LEDs (OLEDs). The embodiment is not limited thereto, and for example, if the wearable device (101) includes a lens for transmitting external light (or ambient light), the display (220) may include a projector (or projection assembly) for projecting light onto the lens. In one embodiment, the display (220) may be referred to as a display panel and / or a display module. The pixels included in the display (220) may be arranged to face either of the user's two eyes when the wearable device (101) is worn by the user (e.g., the user (110) of FIG. 1). For example, the display (220) may include display areas (or active areas) corresponding to each of the user's two eyes.

[0053] In one embodiment, the camera (225) of the wearable device (101) may include an optical sensor (e.g., a charged coupled device (CCD) sensor, a complementary metal oxide semiconductor (CMOS) sensor) that generates an electrical signal representing the color and / or brightness of light. The camera (225) may be referred to as an image sensor and may be included in the sensor (230) of FIG. 2. The plurality of optical sensors included in the camera (225) may be arranged in the form of a two-dimensional grid (2-dimensional array). The camera (225) may acquire electrical signals of each of the plurality of optical sensors substantially simultaneously, and generate two-dimensional frame data corresponding to light reaching the optical sensors of the two-dimensional grid. For example, photographic data captured using the camera (225) may mean one (a) two-dimensional frame data acquired from the camera (225). For example, video data captured using a camera (225) may mean a sequence of multiple two-dimensional frame data obtained from the camera (225) according to a frame rate.

[0054] According to one embodiment, the wearable device (101) may include a plurality of cameras, as examples of cameras (225), arranged in different directions. The plurality of cameras may include a gaze tracking camera (225-1) configured to be arranged toward the eyes of a user wearing the wearable device (101). The plurality of cameras may include an external camera (225-2) configured to be arranged toward the external environment when the wearable device (101) is worn by a user. The processor (210) may identify the direction of the user's gaze using images and / or videos acquired from the gaze tracking camera (225-1). The gaze tracking camera may include an infrared (IR) sensor. The gaze tracking camera may be referred to as an eye sensor and / or an eye tracker.

[0055] For example, the processor (210) can identify an external object using an image and / or video acquired from an external camera (225-2). For example, the processor (210) can identify a position, shape, and / or gesture (e.g., a hand gesture) of a hand of a user wearing the wearable device (101) based on an image and / or video acquired from the external camera (225-2). Using an image and / or video of the external environment acquired from the external camera (225-2), the processor (210) can recognize or track one or more objects within the external environment.

[0056] According to one embodiment, a sensor (230) of a wearable device (101) may generate electrical information from non-electronic information related to the wearable device (101), which may be processed and / or stored by a processor (210) and / or a memory (215) of the wearable device (101). The information may be referred to as sensor data. The sensor (230) may include a global positioning system (GPS) sensor for detecting a geographic location of the wearable device (101), an image sensor (e.g., a camera (225)), an audio sensor (e.g., a microphone and / or a microphone array including multiple microphones), an ambient light sensor, an inertial measurement unit (IMU) (e.g., an acceleration sensor, a gyro sensor, and / or a geomagnetic sensor), a light detection and ranging (LiDAR) sensor, and / or a time-of-flight (ToF) sensor (or ToF camera). Like the ToF sensor, the wearable device (101) may include a sensor configured to detect a distance between the wearable device (101) and an external object. A sensor that detects a distance between external objects may be referred to as a depth sensor.

[0057] In one embodiment, the depth sensor included in the wearable device (101) may include a ToF sensor and / or a structured light (SL) sensor. The SL sensor may be referred to as an SL camera. The ToF sensor may be referred to as a ToF camera. The SL sensor may emit or output a light pattern (e.g., a plurality of dots) of a specific wavelength (e.g., an infrared wavelength). When an external object reflects the light pattern, the light pattern may be distorted by embossing on the surface of the external object. By detecting the reflected light for the light pattern, the SL sensor and / or the processor (210) connected to the SL sensor may recognize the distortion. Using the distortion, the processor (210) may calculate the distance between the wearable device (101) and the external object. The ToF sensor may emit light of a specific wavelength (e.g., an infrared wavelength) in units of nanoseconds. The ToF sensor can measure the time it takes for light reflected by an external object to propagate to the ToF sensor. Using the measured time, the ToF sensor and / or processor (210) can calculate or determine the distance between the external object and the wearable device (101). Information calculated using the ToF sensor and / or the SL sensor can be referred to as a depth map.

[0058] In one embodiment, the communication circuit (235) of the wearable device (101) may include circuitry for supporting transmission and / or reception of signals between the wearable device (101) and an external electronic device. The communication circuit (235) may include, for example, at least one of a modem, an antenna, and an optical / electronic (O / E) converter. The communication circuit (235) may support transmission and / or reception of information based on various protocols, such as Ethernet, a local area network (LAN), a wide area network (WAN), wireless fidelity (WiFi), Bluetooth, Bluetooth low energy (BLE), ZigBee, long term evolution (LTE), 5G new radio (NR), 6G, and / or above-6G. In one embodiment, the communication circuit (235) may be referred to as a communication processor and / or a communication module.

[0059] According to one embodiment, instructions (or commands) to be input to the processor (210) may be stored in the memory (215) of the wearable device (101). The instructions may include binary codes representing functions, operations, and / or actions to be executed by the processor (210). A set of instructions may include a program, firmware, a library, an operating system, a process, a routine, a sub-routine, and / or a software application (hereinafter, “application”). For example, the wearable device (101) and / or the processor (210) may perform at least one of the operations of FIG. 3, FIG. 5, FIG. 8, and / or FIG. 11 when executing the instructions. Hereinafter, the fact that a software application is installed in a wearable device (101) may mean that instructions provided in the form of a software application (or package) are stored in a memory (215), and that the software application is stored in a format executable by the processor (210) (e.g., a file with an extension specified by the operating system of the wearable device (101)).

[0060] Referring to FIG. 2, programs installed in the wearable device (101) may be included in any one of different layers, including the application layer (240), the framework layer (250), and / or the hardware abstraction layer (HAL) (280), based on the target. For example, the hardware abstraction layer (280) may include programs (e.g., modules or drivers) designed to target the hardware of the wearable device (101) (e.g., the display (220), the camera (225), the sensor (230), and / or the communication circuit (235)). The framework layer (250) may be referred to as an XR framework layer from the perspective of including one or more programs for providing an XR (extended reality) service. For example, the layers illustrated in FIG. 2 may be logically (or for convenience of explanation) separated, and may not mean that the address space of the memory (215) is separated by the layers.

[0061] For example, within the framework layer (250), programs designed to target at least one of the hardware abstraction layer (280) and / or the application layer (240) (e.g., a position tracker (271), a space recognizer (272), a gesture tracker (273), an eye-gaze tracker (274), and / or a face tracker (275)) may be included. The programs included in the framework layer (250) may provide an application programming interface (API) that is executable (or callable) based on other programs.

[0062] For example, the application layer (240) may include a program designed to target users of the wearable device (101). As an example of programs included in the application layer (240), an extended reality (XR) system user interface (UI) (241) and / or an XR application (242) are exemplified, but the embodiment is not limited thereto. For example, programs (e.g., software applications) included in the application layer (240) may call an API to cause execution of functions supported by programs included in the framework layer (250).

[0063] For example, the wearable device (101) may display one or more visual objects on the display (220) for performing interaction with a user based on the execution of the XR system UI (241). A visual object may refer to an object that can be placed on a screen for transmitting information and / or interacting with a user, such as text, an image, an icon, a video, a button, a checkbox, a radio button, a text box, a slider, and / or a table. A visual object may be referred to as a visual guide, a virtual object, a visual element, a UI element, a view object, and / or a view element. The wearable device (101) may provide the user with functions available within a virtual space based on the execution of the XR system UI (241).

[0064] Referring to FIG. 2, a lightweight renderer (243) and / or an XR plug-in (244) are illustrated to be included within the XR system UI (241), but are not limited thereto. For example, based on the XR system UI (241), the processor (210) may execute a lightweight renderer (243) and / or an XR plug-in (244) within the framework layer (250).

[0065] For example, the wearable device (101) may obtain resources (e.g., APIs, system processes, and / or libraries) used to define, create, and / or execute a rendering pipeline that allows partial changes based on the execution of a lightweight renderer (243). The lightweight renderer (243) may be referred to as a lightweight render pipeline in terms of defining a rendering pipeline that allows partial changes. The lightweight renderer (243) may include a renderer that is built prior to the execution of a software application (e.g., a prebuilt renderer). For example, the wearable device (101) may obtain resources (e.g., APIs, system processes, and / or libraries) used to define, create, and / or execute an entire rendering pipeline based on the execution of an XR plug-in (244). The XR plugin (244) can be referred to as an open XR native client from the perspective of defining (or setting up) the entire rendering pipeline.

[0066] For example, the wearable device (101) may display a screen representing at least a portion of a virtual space on the display (220) based on the execution of the XR application (242). The XR plug-in (244-1) included in the XR application (242) may include instructions that support functions similar to those of the XR plug-in (244) of the XR system UI (241). Descriptions of the XR plug-in (244-1) that overlap with those of the XR plug-in (244) may be omitted. The wearable device (101) may cause the execution of the virtual space manager (251) based on the execution of the XR application (242).

[0067] According to one embodiment, the wearable device (101) may provide a virtual space service based on the execution of the virtual space manager (251). For example, the virtual space manager (251) may include a platform for supporting the virtual space service. The wearable device (101) may identify a virtual space formed based on the user's location indicated by data acquired through the sensor (230) based on the execution of the virtual space manager (251), and may display at least a portion of the virtual space on the display (220). The virtual space manager (251) may be referred to as a composition presentation manager (CPM).

[0068] For example, the virtual space manager (251) may include a runtime service (252). As an example, the runtime service (252) may be referred to as an OpenXR runtime module (or an OpenXR runtime program). The wearable device (101) may execute at least one of a user's pose prediction function, a frame timing function, and / or a spatial input function based on the execution of the runtime service (252). As an example, the wearable device (101) may perform rendering for a virtual space service to the user based on the execution of the runtime service (252). For example, a function related to a virtual space, executable by the application layer (240), may be supported based on the execution of the runtime service (252).

[0069] For example, the virtual space manager (251) may include a pass-through manager (253). Based on the execution of the pass-through manager (253), the wearable device (101) may display an image and / or video representing an actual space acquired through an external camera (225-2) on at least a portion of the screen while displaying a screen representing a virtual space on the display (220).

[0070] For example, the virtual space manager (251) may include an input manager (254). The wearable device (101) may identify data (e.g., sensor data) acquired by executing one or more programs included in the recognition service layer (270) based on the execution of the input manager (254). The wearable device (101) may use the acquired data to identify a user input related to the wearable device (101). The user input may be related to a motion (e.g., a hand gesture), a gaze, and / or a speech of the user identified by the sensor (230) and / or the camera (225) (e.g., an external camera (225-2). The user input may be identified based on an external electronic device connected (or paired) via the communication circuit (235).

[0071] For example, the perception abstract layer (260) can be used for data exchange between the virtual space manager (251) and the perception service layer (270). From the perspective of being used for data exchange between the virtual space manager (251) and the perception service layer (270), the perception abstract layer (260) can be referred to as an interface. For example, the perception abstract layer (260) can be referenced as OpenPX. The perception abstract layer (260) can be used for a perception client and a perception service.

[0072] According to one embodiment, the recognition service layer (270) may include one or more programs for processing data acquired from a sensor (230) and / or a camera (225). The one or more programs may include at least one of a position tracker (271), a space recognizer (272), a gesture tracker (273), and / or an eye tracker (274). The type and / or number of the one or more programs included in the recognition service layer (270) are not limited to those illustrated in FIG. 2.

[0073] For example, the wearable device (101) can identify the pose of the wearable device (101) using the sensor (230) based on the execution of the position tracker (271). The wearable device (101) can identify the 6 degrees of freedom pose (6 dof pose) of the wearable device (101) using data acquired using the external camera (225-2) and / or IMU (e.g., a gyro sensor, an acceleration sensor, and / or a geomagnetic sensor) based on the execution of the position tracker (271). The position tracker (271) may be referred to as a head tracking (HeT) module (or head tracker, a head tracking program).

[0074] For example, the wearable device (101) may obtain information for providing a three-dimensional virtual space corresponding to the surrounding environment (e.g., external space) of the wearable device (101) (or the user of the wearable device (101)) based on the execution of the space recognizer (272). The wearable device (101) may reproduce the surrounding environment of the wearable device (101) in three dimensions using data obtained using an external camera (e.g., camera (225)) based on the execution of the space recognizer (272). The wearable device (101) may identify at least one of a plane, a slope, and stairs based on the surrounding environment of the wearable device (101) reproduced in three dimensions based on the execution of the space recognizer (272). The space recognizer (272) may be referred to as a scene understanding (SU) module (or a scene recognition program).

[0075] For example, the wearable device (101) may identify (or recognize) a pose and / or gesture of a hand of a user of the wearable device (101) based on the execution of the gesture tracker (273). As an example, the wearable device (101) may identify a pose and / or gesture of a hand of a user using data acquired from an external camera (e.g., camera (225)) based on the execution of the gesture tracker (273). As an example, the wearable device (101) may identify a pose and / or gesture of a hand of a user based on data (or images) acquired using an external camera based on the execution of the gesture tracker (273). The gesture tracker (273) may be referred to as a hand tracking (HaT) module (or hand tracking program) and / or a gesture tracking module.

[0076] For example, the wearable device (101) can identify (or track) eye movements of a user of the wearable device (101) based on the execution of the gaze tracker (274). As an example, the wearable device (101) can identify eye movements of the user using data acquired from the gaze tracking camera (225-1) based on the execution of the gaze tracker (274). The gaze tracker (274) may be referred to as an eye tracking (ET) module (or eye tracking program) and / or a gaze tracking module.

[0077] For example, the recognition service layer (270) of the wearable device (101) may further include a face tracker (275) for tracking the user's face. For example, the wearable device (101) may identify (or track) the movement of the user's face and / or the user's expression based on the execution of the face tracker (275). The wearable device (101) may estimate the user's expression based on the movement of the user's face based on the execution of the face tracker (275). As an example, the wearable device (101) may identify the movement of the user's face and / or the user's expression based on data (e.g., images and / or videos) acquired using a camera (225) (e.g., a camera directed at at least a portion of the user's face) based on the execution of the face tracker (275).

[0078] In one embodiment, the processor (210) may obtain information about the external environment using the camera (225) and / or the sensor (230). The information may indicate the geographic location of the wearable device (101). For example, the information may include GPS coordinates of the wearable device (101). The wearable device (101) may compare the geographic location of the wearable device (101) with a safety zone (e.g., the safety zone (130) of FIG. 1) to determine whether the wearable device (101) is moved beyond the safety zone. For example, the wearable device (101) may determine the boundary of the safety zone passed by the wearable device (101) at the time of moving beyond the safety zone. When the wearable device (101) is moved outside the safety zone, the wearable device (101) can display images and / or videos obtained from the camera (225) and representing a view of the external environment on at least a portion of the display (220).

[0079] For example, a safety zone configured to provide a virtual space may be located away from obstacles that impede the user's movement within the external environment. At a first boundary of the safety zone adjacent to an obstacle, the wearable device (101) may output a relatively high-level warning to prevent a collision (or an accident caused by the collision) between the user wearing the wearable device (101) and the obstacle. At a second boundary of the safety zone located away from the obstacle, the wearable device (101) may output a relatively low-level warning to the user wearing the wearable device (101). At the second boundary and / or beyond the second boundary, the wearable device (101) may display images and / or videos acquired through the camera (225) on the display (220) to support interaction between the external environment and the user. Images and / or videos being acquired through the camera (225) may be provided based on a partial pass-through mode.

[0080] For example, a warning output at the boundary of a safety zone may be related to characteristics of the external environment beyond the boundary. At the boundary of the safety zone and / or beyond the boundary, the wearable device (101) may determine or change properties of the pass-through mode (e.g., width, height, position within the display area, and / or size of an image and / or video displayed for pass-through and acquired from an external camera (225-2)) depending on the shape of the virtual space.

[0081] Hereinafter, with reference to FIG. 3, exemplary operations of a wearable device (101) and / or a processor (210) for setting a safety zone are described.

[0082] FIG. 3 illustrates a flowchart of a wearable device according to one embodiment. The wearable device (101) of FIG. 1 and / or FIG. 2, and / or the processor (210) of FIG. 2, may perform the operations of the wearable device described with reference to FIG. 3. The operations of FIG. 3 may be performed based on the wearable device executing the program (e.g., the pass-through manager (253)) described with reference to FIG. 2. The order in which the operations of FIG. 3 are performed is not limited to the order illustrated in FIG. 3. For example, the processor of the wearable device may perform the operations of FIG. 3 in a different order than the order illustrated in FIG. 3. For example, the processor of the wearable device may perform at least two of the operations of FIG. 3 substantially simultaneously.

[0083] Referring to FIG. 3, in operation (310), according to one embodiment, a processor of a wearable device may receive an input for setting a safety zone (e.g., safety zone (130) of FIG. 1) for the safety of a user wearing the wearable device (e.g., user (110) of FIG. 1). The input may be identified or detected based on a signal received from an external electronic device connected to the wearable device (e.g., a remote controller wirelessly connected via a communication circuit (235) of FIG. 2). The input may be identified or detected using data from a camera (e.g., gaze tracking camera (225-1) of FIG. 2) and / or a sensor (e.g., sensor (230) of FIG. 2) of the wearable device. For example, the processor may detect or receive an input of the action (310) using the gaze position of a user wearing the wearable device recognized through an eye tracking camera. For example, the processor may detect or receive an input of the action (310) using a hand gesture recognized through an external camera (e.g., the external camera (225-2) of FIG. 2).

[0084] Referring to FIG. 3, within operation (320), the processor of the wearable device according to one embodiment may determine whether an external object is detected that is less than a specified distance from the boundary of the safety zone. If an external object is detected that is less than a specified distance from the boundary of the safety zone (320 - Yes), the processor may perform operation (330). If an external object is detected that is more than a specified distance from the boundary of the safety zone, or no external object is detected (320 - No), the processor may perform operation (340). The specified distance in operation (320) may be empirically determined such that the external object does not collide with or come into contact with the user wearing the wearable device before the switching from VR mode to pass-through mode is completed when moving outside the safety zone. The specified distance of the movement (320) may be predetermined by a program (e.g., an operating system and / or software application) running on the wearable device.

[0085] Referring to FIG. 3, in operation (330), a processor of a wearable device according to one embodiment may determine a portion of a boundary of a safety zone adjacent to an external object as a first-type boundary, which is set to stop providing a VR environment. An external object that may collide with a user may be located in an external space less than a specified distance from the first-type boundary. The processor, having identified the first-type boundary, may store the result of identifying the first-type boundary in a memory (e.g., memory (215) of FIG. 2).

[0086] Referring to FIG. 3, in operation (340), the processor of the wearable device according to one embodiment may check or determine whether a user wearing the wearable device is capable of moving beyond a specified distance from the boundary of a safety zone. For example, the processor may detect or determine an empty space beyond the boundary (e.g., a space having a width and / or height exceeding the specified distance). If the user wearing the wearable device is capable of moving beyond the specified distance from the boundary of the safety zone (340—Yes), the processor may perform operation (350). If the user wearing the wearable device is capable of moving less than or equal to the specified distance from the boundary of the safety zone (340—No), the processor may perform operation (340).

[0087] Referring to FIG. 3, in operation (350), according to one embodiment, a processor of a wearable device may set a portion of the boundary of a safety zone as a second type boundary. The second type boundary may be a remaining portion of the boundary of the safety zone that is different from the first type boundary. The second type boundary may be set to at least temporarily maintain the provision of a virtual space. The second type boundary may be set to visualize information about the external environment while maintaining a user experience based on the virtual space beyond the boundary. There may not be any external objects that may collide with the user in the external space within a specified distance from the second type boundary. The processor that has confirmed the second type boundary may store the result of confirming the second type boundary in memory.

[0088] Referring to FIG. 3, after receiving an input of an action (310), the boundary of a safety zone indicated by the input can be classified into a first type boundary and a second type boundary. Information indicating segments of the classified boundary and a type corresponding to each of the segments can be stored in the memory of the wearable device. The processor can display a UI notifying movement of the wearable device toward the boundary, depending on the type of boundary indicated by the information, based on detecting the position of the wearable device moving toward the boundary.

[0089] Below, with reference to FIG. 4, an exemplary operation of a wearable device that receives input of an action (310) is described.

[0090] FIG. 4 illustrates an exemplary operation of a wearable device (101) that receives an input for setting a safety zone configured to provide a virtual space. The wearable device (101) of FIGS. 1 to 2 and / or the processor (210) of FIG. 2 may perform the operation of the wearable device (101) described with reference to FIG. 4. The operation of the wearable device (101) described with reference to FIG. 4 may be related to at least one of the operations of FIG. 3, or may be performed similarly.

[0091] Referring to FIG. 4, a wearable device (101) can be wirelessly connected to a remote controller (410). The wireless connection between the wearable device (101) and the remote controller (410) can be established through a communication circuit (e.g., the communication circuit (235) of FIG. 2). In one embodiment, the wearable device (101) can receive an input to start setting a safety zone for providing a virtual space. The input can be received through a menu provided by the wearable device (101) (e.g., a settings application and / or any software application executed by the wearable device (101). Referring to FIG. 4, the wearable device (101) that has received an input to start setting a safety zone can enter a mode for setting a safety zone.

[0092] Referring to FIG. 4, an exemplary state of a wearable device (101) receiving a signal representing an input for setting a safety zone from a remote controller (410) is illustrated. The wearable device (101) can display a visual object (412) corresponding to the remote controller (410) within a screen (401). Using a signal received from the remote controller (410) (e.g., a position and / or direction of the remote controller (410), the wearable device (101) can display a virtual line (414) extending from the visual object (412) within the screen (401). The line (414) can be referred to as a ray, a pointer, and / or a cursor.

[0093] According to one embodiment, the wearable device (101) may receive an input for intersecting a line (414) on a plane (e.g., a floor surface of an external environment) expressed through the screen (401) based on a signal received from the remote controller (410). The wearable device (101) that has received the input may display, within the screen (401), a line (420) expressing a path (or trajectory) of the line (414) intersecting the plane. Referring to FIG. 4, the wearable device (101) that has confirmed a line (414) intersecting at a position (g1) on the plane may display, within the screen (401), a line (420) connecting the intersection points between the plane and the line (414) from the position (g1) to the position (g1). When the shape of the line (420) has the shape of a closed curve on a plane, the wearable device (101) can determine or set the area inside the line (420) as a safety zone configured to provide a virtual space.

[0094] The wearable device (101) that has identified the safety zone specified by the line (420) can determine the properties of the safety zone. The properties of the safety zone can be determined for at least a portion of the boundary (or boundary surface) of the safety zone. Referring to FIG. 4, the boundary surface of the safety zone can be a virtual curved surface (or plane) extending in the direction of the +z axis from the line (420). The wearable device (101) can determine the properties of the safety zone by using the distance between the boundary surface and an external object located outside the safety zone.

[0095] Referring to FIG. 4, portions of the boundary surface of the safety zone are illustrated in the form of a grid. The pattern (or texture) of each portion may represent an attribute of the portion determined by the wearable device (101). For example, the wearable device (101) that detects a bookshelf (123) less than a specified distance from the boundary of the safety zone may set portions of the boundary surface of the safety zone less than the specified distance from the bookshelf (123) as a boundary of the first type (e.g., the first type of operation (330) of FIG. 3). For example, when the distance between the boundary surface of the safety zone and the lamp (122) is less than the specified distance, the wearable device (101) may set portions of the boundary surface of the safety zone adjacent to the lamp (122) as a boundary of the first type.

[0096] Referring to FIG. 4, an empty space may be formed outside the safety zone, above the sofa (121). The wearable device (101) may set portions of the boundary surface of the safety zone adjacent to the sofa (121) as a first type boundary. The wearable device (101) may set portions of the boundary surface of the safety zone adjacent to the empty space formed above the sofa (121) as a second type boundary (e.g., the second type of operation (350) of FIG. 3).

[0097] Referring to FIG. 4, an empty space (e.g., a space in which a user (110) can move) may be formed outside the safety zone in the direction of the + x-axis from the sofa (121). The wearable device (101) may set portions of the boundary surface of the safety zone adjacent to the empty space as a second type boundary. After setting the exemplary safety zone described with reference to FIG. 4, the wearable device (101) may output a notification related to the boundary of the safety zone. Hereinafter, an exemplary operation of the wearable device (101) that outputs a notification related to the boundary of the safety zone is described with reference to FIGS. 5, 6, and 7A to 7D.

[0098] FIG. 5 illustrates an exemplary flowchart of the operation of a wearable device providing a virtual space. The wearable device (101) of FIG. 1 and / or FIG. 2, and / or the processor (210) of FIG. 2, may perform the operation of the wearable device described with reference to FIG. 5. The operation of FIG. 5 may be performed based on the wearable device executing the program described with reference to FIG. 2 (e.g., the pass-through manager (253), the space recognizer (272), and / or the location tracker (271)). The order in which the operations of FIG. 5 are performed is not limited to the order illustrated in FIG. 5. For example, the processor of the wearable device may perform the operations of FIG. 5 in a different order than the order illustrated in FIG. 5. For example, the processor of the wearable device may perform at least two of the operations of FIG. 5 substantially simultaneously. The operation of FIG. 5 may be related to the safety zone established by the exemplary operation of FIG. 3. For example, after the safety zone is established, the processor of the wearable device may perform the operation of FIG. 5.

[0099] Referring to FIG. 5, within operation (510), a processor of a wearable device according to one embodiment may provide a virtual space using displays (e.g., display (220) of FIG. 2). For example, while located within a safety zone configured to provide a virtual space, the processor may perform operation (510) to provide the virtual space. Screens (181, 182, 184) of FIG. 1 may be displayed on displays controlled by the processor performing operation (510).

[0100] Referring to FIG. 5 , in operation (520), a processor of a wearable device according to one embodiment may determine or verify whether a distance between the wearable device and / or a designated body part and a boundary of a safety zone is less than a threshold distance. The processor may perform operation (520) using various distances to the safety zone. For example, the processor may compare or calculate a distance between a location of the wearable device and a boundary of the safety zone, as detected using the sensor (230) and / or the external camera (225-2) of FIG. 2 . For example, the processor may calculate or determine a distance between a location of a designated body part (e.g., a hand) of a user wearing the wearable device (e.g., the user (110) of FIG. 1 ) and a boundary of the safety zone, as detected using the sensor (230) and / or the external camera (225-2) of FIG. 2 . For example, the processor can determine or identify the distance between the location of an external electronic device (e.g., a remote controller (410) of FIG. 4) connected via the communication circuit (235) of FIG. 2 and the boundary of a safety zone. The distances illustrated above are exemplarily described with reference to FIG. 6.

[0101] Within operation (520), if the wearable device and / or the designated body part are spaced less than a threshold distance from the boundary of the safety zone (520 - Yes), the processor may perform operation (530). If the wearable device and / or the designated body part are spaced more than a threshold distance from the boundary of the safety zone (520 - No), the processor may not execute the function of operation (530) and may continue to provide the virtual space of operation (510). For example, while inside the safety zone and spaced more than a threshold distance from the boundary of the safety zone, the processor may continue to provide the virtual space of operation (510).

[0102] Referring to FIG. 5, in operation (530), according to one embodiment, the processor of the wearable device may execute a function for notifying an approach to the boundary, depending on the type of the portion of the boundary that is less than the threshold distance. The type of operation (530) may include the types set for each portion of the boundary surface described with reference to FIGS. 3 and 4. The type of operation (530) may be related to a risk of collision (or risk of accident) that may occur when moving outside the safety zone. For example, the type of operation (530) may include a first type for indicating a boundary with a relatively high risk of collision, and a second type for indicating a boundary with a relatively low risk of collision.

[0103] For example, if a wearable device is detected moving toward a boundary of a first type, the processor may output an alarm (or warning) corresponding to the first type. If a wearable device is detected moving toward a boundary of a second type, the processor may output a notification corresponding to the second type. The alarm corresponding to the first type may include at least one of a virtual object of a first size having a first designated color, such as red, a haptic feedback (e.g., vibration) of a first intensity, and an acoustic signal of a first volume. The notification corresponding to the second type may include at least one of a virtual object of a second size (e.g., smaller than the first size) having a second designated color (e.g., green and / or blue), a haptic feedback (e.g., vibration) of a second intensity (e.g., smaller than the first intensity), and an acoustic signal of a second volume (e.g., smaller than the first volume).

[0104] FIG. 6 illustrates exemplary reference areas (e.g., reference areas (610, 620, 630)) formed to determine whether a wearable device (101) approaches the boundary of a safety zone. The wearable device (101) of FIG. 1 and / or FIG. 2, and / or the processor (210) of FIG. 2, may perform the operation of the wearable device (101) described with reference to FIG. 6. The operation of FIG. 6 may be performed based on the wearable device (101) executing a program (e.g., a pass-through manager (253) and / or a location tracker (271)) described with reference to FIG. 2.

[0105] According to one embodiment, the wearable device (101) can detect the location of the wearable device (101) and / or calculate the distance from the boundary of a safety zone using a sensor (e.g., sensor (230) of FIG. 2). For example, the wearable device (101) can calculate the location within the external environment and / or the distance from the boundary of a safety zone using coordinates (e.g., GPS coordinates) indicating the location of the wearable device (101) detected by the sensor.

[0106] For example, the wearable device (101) can set or determine a reference area (610) that includes the body of a user (110) wearing the wearable device (101). The reference area (610) can be a three-dimensional virtual space formed based on the position of the wearable device (101). Although the reference area (610) having an ellipsoidal shape is illustrated as an example, the embodiment is not limited thereto.

[0107] For example, the wearable device (101) can set a reference area (620) including the remote controller (410). The reference area (620) can be a three-dimensional virtual space formed based on the position of the remote controller (410). The wearable device (101) can identify or determine the reference area (620) using the GPS coordinates of the remote controller (410) received from the remote controller (410) through the communication circuit (235) of FIG. 2. The reference area (620) can have a spherical shape based on the remote controller (410). The embodiment is not limited thereto.

[0108] For example, the wearable device (101) can set a reference area (630) including a hand (e.g., a right hand (110-r)) of a user wearing the wearable device (101). The reference area (630) can be set based on, or generated from, the position of the right hand (110-2) detected using the external camera (225-2) of FIG. 2 (or a sensor (230) of FIG. 2, such as a lidar sensor). The reference area (630) can have a shape of a sphere (or ellipsoid) including the right hand (110-r). The embodiment is not limited thereto.

[0109] In one embodiment, wherein at least one reference area (e.g., reference areas (610, 620, 630)) associated with a wearable device (101) and / or a user (110) wearing the wearable device (101) is set, the wearable device (101) may use the positional relationship between the boundary of the safety zone and the reference area to determine whether to output a notification related to the boundary of the safety zone. For example, the wearable device (101) may use an algorithm such as a collision test to check or determine whether each of the reference areas (610, 620, 630) intersects (or overlaps) the boundary of the safety zone. In one embodiment of FIG. 6, when the boundary surface of the safety zone intersects at least one of the reference areas (610, 620, 630), the wearable device (101) may output a notification based on the type (e.g., first type and / or second type) of the portion of the boundary surface intersecting at least one of the reference areas (610, 620, 630).

[0110] Hereinafter, with reference to FIGS. 7A to 7D, an exemplary UI displayed by a wearable device (101) is described when at least one of the reference areas (610, 620, 630) intersects the boundary surface of a safety zone. The UI may be provided differentially depending on the risk of collision between an external object located outside the safety zone and a user (110).

[0111] FIGS. 7A, 7B, 7C, and 7D illustrate exemplary screens (701, 702, 703, 704) displayed by a wearable device (101) to indicate the boundaries of a safety zone. The wearable device (101) of FIGS. 1 to 2 and / or the processor (210) of FIG. 2 may perform the operations of the wearable device (101) described with reference to FIGS. 7A to 7D. The operations of the wearable device (101) described with reference to FIGS. 7A to 7D may be related to at least one of the operations of FIGS. 3 and / or 5.

[0112] Referring to FIG. 7A, an exemplary screen (701) displayed by a wearable device (101) positioned inside a safety zone (710) is illustrated. In one embodiment including displays configured to be positioned toward the eyes of a user (110) wearing the wearable device (101), the wearable device (101) may display the screen (701) on at least one of the displays. The safety zone (710) may have the shape of a closed curve of line (420), as described with reference to FIG. 4. The safety zone (710) may have properties determined based on the operation described with reference to FIG. 4.

[0113] Referring to FIG. 7A, while positioned inside a safety zone (710), the wearable device (101) may display a screen (701) including images and / or videos representing at least a portion of a virtual space. In order to provide a VR-based user experience, the wearable device (101) may display a virtual space having a layout independent of the external environment on the entire screen (701).

[0114] While displaying the screen (701), the wearable device (101) may use a sensor (e.g., sensor (230) of FIG. 2) to identify the location of the wearable device (101) and / or movement of the wearable device (101). When changing the virtual space in conjunction with the location of the wearable device (101), the wearable device (101) may display the screen (701) including a view of the virtual space as seen from a location within the virtual space that is in conjunction with the location of the wearable device (101). In the exemplary state of FIG. 7A, when the head of the user (110) wearing the wearable device (101) is rotated toward the right, the wearable device (101) may display another part of the virtual space corresponding to the right side of the part of the virtual space that was displayed before the rotation within the screen (701). In the exemplary state of FIG. 7a, when a user (110) wearing a wearable device (101) moves, the wearable device (101) can display a portion of a virtual space mapped to the position of the moved user (110) within the screen (701).

[0115] A user (110) viewing a virtual space included in a screen (701) can move according to the perspective of the virtual space. Referring to FIG. 7B, a user (110) recognizing the virtual space can move to a location (p1) adjacent to the boundary of a safety zone (710). At the location (p1) adjacent to the boundary of the safety zone (710), the wearable device (101) can display a screen (702). The screen (702) can include an image and / or video of the virtual space viewed from a location within the virtual space linked to the location (p1).

[0116] Referring to an exemplary screen (702) of FIG. 7B, based on identifying a wearable device (101) moving toward a portion of the boundary of the safety zone (710) from the inside of the safety zone (710), the wearable device (101) may provide a virtual plane (721) representing a portion to which the wearable device (101) is approaching, using displays. As described above with reference to FIG. 4, since the bookshelf (123) is spaced apart from the boundary of the safety zone (710) adjacent to the location (p1) by a distance less than a specified distance, the wearable device (101) may determine the portion of the boundary of the safety zone (710) adjacent to the location (p1) as a boundary of the first type. At a location (p1) adjacent to a boundary of the first type, the wearable device (101) may display a virtual plane (721) to indicate movement of the wearable device (101) toward the boundary of the first type.

[0117] For example, the wearable device (101) may use the displays to provide a virtual plane (721) having a color (e.g., a first designated color including red) determined using information indicating whether a portion of the safety zone (710) adjacent to a location (p1) can be moved out of the safety zone (710). The virtual plane (721) may have binocular disparity and / or depth values ​​based on a distance between the location (p1) and the boundary of the safety zone (710). For example, the wearable device (101) may use the displays to display the virtual plane (721) such that a user (110) at the location (p1) has a perspective similar to that of viewing the boundary of the safety zone (710) from the location (p1). For example, the binocular disparity of the virtual plane (721) included in the screen (702) can be set based on the distance between the location (p1) and the boundary surface of the safety zone (710) adjacent to the location (p1) (e.g., a portion of the boundary surface between the bookshelf (123) and the user (110)).

[0118] Referring to FIG. 7B, in an exemplary state indicating movement of the wearable device (101) to a first type boundary, the wearable device (101) may display a virtual plane (721) having a first size and a first designated color. The first size of the virtual plane (721) may be predetermined, or may be determined according to a distance between the location (p1) and the first type boundary. For example, the first designated color may be determined as a designated primary color, such as red. For example, the first designated color may be determined according to a color distribution of pixels of displays controlled to display the screen (702). For example, the first designated color may be determined as a complementary color of an average color represented by the color distribution.

[0119] In one embodiment, the wearable device (101) that detects movement of the wearable device (101) toward a first type boundary may control a haptic actuator (or vibration motor) to provide vibration feedback of a first intensity. In response to detecting movement of the wearable device (101) toward the first type boundary, the wearable device (101) may play an acoustic signal of a first volume (e.g., an acoustic signal having a designated melody recorded thereon for alerting). The wearable device (101) that detects movement of the wearable device (101) toward the first type boundary may output the virtual plane (721), the vibration feedback of the first intensity, the acoustic signal of the first volume, or any combination thereof as a UI for alerting the movement of the wearable device (101).

[0120] Referring to FIG. 7c, a user (110) who recognizes the virtual space can move to a location (p2) adjacent to the boundary of the safety zone (710). At the location (p2) adjacent to the boundary of the safety zone (710), the wearable device (101) can display a screen (703). The screen (703) can include an image and / or video representing a view of the virtual space as seen from a location within the virtual space mapped to the location (p2).

[0121] Within the exemplary screen (703) of FIG. 7c, the wearable device (101) may display a virtual plane (731) to indicate movement of the wearable device (101) toward the boundary of the safety zone (710), together with a view of the virtual space. As described above with reference to FIG. 4, since an empty space in which the user (110) can move is formed outside the safety zone (710) adjacent to the location (p2), the wearable device (101) may set a portion of the boundary of the safety zone (710) adjacent to the location (p2) as a boundary of the second type. At the location (p2) adjacent to the boundary of the second type, the wearable device (101) may display a virtual plane (731) to indicate movement of the wearable device (101) toward the boundary of the second type.

[0122] For example, the wearable device (101) can determine the properties (e.g., location, size, transparency, radius, diameter, and / or color) of the virtual plane (731) using the properties assigned to the portion of the boundary of the safety zone (710) adjacent to the location (p2). The location of the virtual plane (731) within the screen (703) can be related to the positional relationship between the location (p2) and the boundary surface of the safety zone (710). To indicate movement of the wearable device (101) to a boundary of a second type, the wearable device (101) can display the virtual plane (731) having a second size and a second designated color. For example, the second size can be predetermined to be smaller than the first size, or can be determined based on the distance between the location (p2) and the boundary of the second type. For example, the second designated color may be determined as a different color (e.g., a designated primary color such as blue) than the first designated color. For example, the second designated color may be determined based on the color distribution of pixels of displays controlled to display the screen (703). A virtual plane (731) displayed using a second designated color that is smaller than the first size and different from the first designated color may have relatively lower visibility than the virtual plane (721) of FIG. 7B having the first size and the first designated color.

[0123] When displaying the virtual plane (721) of FIG. 7B and the virtual plane (731) of FIG. 7C, the wearable device (101) may gradually increase the sizes of the virtual plane (721) and the virtual plane (731) to the first size and the second size, respectively. The speed at which the size of the virtual plane (721) increases may be different from the speed at which the size of the virtual plane (731) increases. For example, the wearable device (101) may determine the speed at which the size of the virtual plane (e.g., the virtual planes (721, 731)) increases, depending on the properties of the boundary. For example, the speed at which the size of the virtual plane (721) corresponding to the first type of boundary increases may be higher than the speed at which the size of the virtual plane (731) corresponding to the second type of boundary increases.

[0124] The embodiment is not limited thereto, and the wearable device (101) may display images and / or videos of the external environment, obtained from a camera (e.g., an external camera (225-2) of FIG. 2), on a virtual plane (731). From the perspective of displaying images and / or videos of the external environment, the virtual plane (731) may be referred to as a partial pass-through object, a penetration area, a transparent area, and / or a window area.

[0125] In one embodiment, the wearable device (101) that detects movement of the wearable device (101) toward a second type boundary may control a haptic actuator (or vibration motor) to provide vibration feedback of a second intensity that is less than the first intensity. The wearable device (101) that detects movement of the wearable device (101) toward a second type boundary may play an acoustic signal of a second volume that is less than the first volume. Based on detecting movement of the wearable device (101) toward a second type boundary, the wearable device (101) may output a virtual plane (731), the vibration feedback of the second intensity, the acoustic signal of the second volume, or any combination thereof, as a UI for notifying movement of the wearable device (101).

[0126] Referring to FIG. 7d, a user (110) who recognizes the virtual space can move to a location (p3) adjacent to the boundary of the safety zone (710). At the location (p3) adjacent to the boundary of the safety zone (710), the wearable device (101) can display a screen (704). The screen (704) can include an image and / or video representing a view of the virtual space as seen from a location within the virtual space linked to the location (p3).

[0127] Within the exemplary screen (704) of FIG. 7D, the wearable device (101) may display a virtual plane (741) for notifying movement of the wearable device (101) toward the boundary of the safety zone (710) together with a view of the virtual space. As described above with reference to FIG. 4, since the sofa (121) is located beyond the boundary of the safety zone (710) adjacent to the location (p3), the wearable device (101) may determine a portion of the boundary of the safety zone (710) adjacent to the sofa (121) as a first type boundary, and may determine another portion of the boundary of the safety zone (710) adjacent to the empty space above the sofa (121) as a second type boundary.

[0128] Referring to FIG. 7d, at a position (p3) adjacent to the boundary of the safety zone (710), the wearable device (101) may display a virtual plane (741) to indicate movement of the wearable device (101) toward the boundary. The virtual plane (741) representing a portion of the boundary surface of the safety zone (710) may have a color and / or a size according to the type of the portion of the boundary surface corresponding to the virtual plane (741). The position of the virtual plane (741) within the screen (704) and / or binocular disparity may be determined based on the positional relationship between the portion of the boundary surface of the safety zone corresponding to the virtual plane (741) and the position (p3).

[0129] Referring to FIG. 7d, a section (b) of a virtual plane (741), corresponding to a portion of the boundary surface of the safety zone (710) adjacent to the sofa (121), may have a first color defined to indicate a boundary of the first type. A section (a) of a virtual plane (741), corresponding to another portion of the boundary surface of the safety zone (710) adjacent to the empty space formed above the sofa (121), may have a second color defined to indicate a boundary of the second type. In the exemplary state of FIG. 7d for indicating all of the boundary surfaces of the first type and the second type using the virtual plane (741), the wearable device (101) may output vibration feedback of an intermediate intensity (or a weighted sum) between the first intensity and the second intensity. In the exemplary state of FIG. 7d, the wearable device (101) can reproduce an audio signal of an intermediate volume (or weighted sum) between the first volume and the second volume. The wearable device (101) can display an image and / or video of the external environment, which is being acquired from the camera, in a section (b) of the virtual plane (741).

[0130] As described above, according to one embodiment, the wearable device (101) may output a notification based on the properties (e.g., collision risk, and / or type) of a portion of the boundary of the safety zone (710) that the user (110) wearing the wearable device (101) is approaching when the user (110) is moving toward the boundary of the safety zone (710). For example, a notification displayed by the wearable device (101) in a first portion of the boundary may be different from a notification displayed by the wearable device (101) in a second portion of the boundary. For example, based on detecting the wearable device (101) moving toward the first boundary of the safety zone (710), the wearable device (101) may provide a first virtual object of a first color (e.g., a virtual plane (721) of FIG. 7B) representing the first boundary using displays. For example, based on detecting a wearable device (101) moving toward a second boundary of a safety zone (720), the wearable device (101) may provide a second virtual object of a second color representing the second boundary (e.g., a virtual plane (731) of FIG. 7C) using displays.

[0131] Hereinafter, with reference to FIGS. 8, 9a, 9b, 9c, 10a, and 10b, an exemplary operation of a wearable device (101) moving from the inside of the safety zone (710) to the outside of the safety zone (710) is described.

[0132] FIG. 8 illustrates a flowchart of a wearable device according to one embodiment. The wearable device (101) of FIG. 1 and / or FIG. 2, and / or the processor (210) of FIG. 2, may perform operations of the wearable device described with reference to FIG. 8. The operations of FIG. 8 may be performed based on the wearable device (101) executing the program (e.g., pass-through manager (253)) described with reference to FIG. 2. The order in which the operations of FIG. 8 are performed is not limited to the order illustrated in FIG. 8. For example, the processor of the wearable device may perform the operations of FIG. 8 in a different order than the order illustrated in FIG. 8. For example, the processor of the wearable device may perform at least two of the operations of FIG. 8 substantially simultaneously. The operations of FIG. 8 may be related to a safety zone established by the exemplary operation of FIG. 3. For example, after a safety zone is established, the processor can perform the operations of FIG. 8.

[0133] Referring to FIG. 8, in operation (810), a processor of a wearable device according to one embodiment may provide a virtual space using displays. The processor may perform operation (810) similarly to operation (510) of FIG. 5. For example, the processor may provide a virtual space to a user wearing the wearable device (e.g., user (110) of FIG. 1). While providing the virtual space of operation (810), the processor may repeatedly perform operation (820).

[0134] Referring to FIG. 8, in operation (820), a processor of a wearable device according to one embodiment may determine or identify whether the wearable device is located within a safety zone (e.g., safety zone (130) of FIG. 1) for the safety of a user wearing the wearable device. The processor may detect the location of the wearable device using sensor data of a sensor (e.g., sensor (230) of FIG. 2). By comparing the detected location with the safety zone, the processor may determine whether the wearable device is located within the safety zone.

[0135] Referring to FIG. 8, while the wearable device is positioned within the safety zone (820 - Yes), the processor may continue to provide the virtual space of operation (810). When detecting the wearable device moving from the inside of the safety zone to the boundary of the safety zone, the processor may continue to provide the virtual space of operation (810) and may execute a function for delimiting the boundary as described above with reference to FIGS. 5, 6, and 7A to 7D. When the wearable device is positioned outside the safety zone (820 - No), the processor may perform operation (830).

[0136] Referring to FIG. 8, in operation (830), according to one embodiment, the processor of the wearable device may determine or identify whether the wearable device has moved out of the safety zone through a first type of boundary of the safety zone. The boundary of the safety zone may have an attribute (e.g., the first type and / or the second type described above with reference to FIG. 3) related to a risk of collision with an external object, based on the operation of the wearable device described above with reference to FIGS. 3 and 4. If an external object is located less than a specified distance from a portion of the boundary of the safety zone, the portion of the boundary may be determined to be a first type boundary. If an external object is located more than a specified distance from a portion of the boundary of the safety zone, or if no external object exists beyond the portion, the portion of the boundary may be determined to be a second type boundary.

[0137] Referring to FIG. 8, if the wearable device is moved out of the safety zone through a first type of boundary (830 - Yes), the processor may perform operation (840). If the wearable device is moved out of the safety zone through a second type of boundary different from the first type (830 - No), the processor may perform operation (850).

[0138] Referring to FIG. 8, in operation (840), a processor of a wearable device according to an embodiment may stop providing a virtual space using displays and display a video acquired using at least one camera (e.g., camera (225) and / or external camera (225-2) of FIG. 2). To stop providing a virtual space, the processor may at least temporarily stop a software application running to provide the virtual space. For example, the processor may change or switch the state of the software application from a foreground state to a background state. When stopping the execution of a software application, the processor may display a virtual object (e.g., an icon) on the displays to resume execution of the software application.

[0139] Within operation (840), the processor may display images and / or videos of the external environment, which are acquired using at least one camera. The processor may display the images and / or videos on the entire display area of ​​the displays. For example, the processor may enter a pass-through mode to provide a view of the external environment through the displays. While displaying the images and / or videos of the external environment, the user may use the images and / or videos to view external objects beyond the first type of boundary. For example, the processor may enter a pass-through mode to display external objects (e.g., obstacles such as walls) adjacent to the user.

[0140] Referring to FIG. 8, in operation (850), according to one embodiment, a processor of a wearable device may determine or identify whether a virtual space is defined at a location of a wearable device that has moved outside a safety zone. The processor may determine or identify whether the virtual space is defined at a location of the wearable device detected by a sensor. For example, the processor may determine whether the virtual space is defined at a current location of the wearable device using a software application that is executed to provide the virtual space. For example, the processor may determine the movability and / or controllability of the wearable device at a location within the virtual space corresponding to the location of the wearable device.

[0141] Within operation (850), the processor may determine whether the virtual space mapped to the safety zone extends outside the safety zone. For example, the processor may determine whether the virtual space provided while the wearable device is positioned inside the safety zone has opened outside the safety zone. If the virtual space extends to the location of the wearable device outside the safety zone, or the virtual space is set to include the location of the wearable device, or the virtual space is defined at the location of the wearable device outside the safety zone (850-Yes), the processor may perform operation (860).

[0142] For example, if the virtual space does not extend to a location of a wearable device outside the safety zone, if the virtual space is set not to include a location of a wearable device, or if the virtual space is not defined at a location of a wearable device outside the safety zone (850-No), the processor may perform operation (840). For example, while providing a virtual space defined within the safety zone, the processor may stop providing the virtual space based on identifying a wearable device that is moved outside the safety zone.

[0143] Referring to FIG. 8, in operation (860), according to one embodiment, a processor of a wearable device may maintain providing a virtual space and display video acquired using at least one camera. The processor may (continuously) display images and / or videos representing the virtual space, as viewed from a location in the virtual space corresponding to a location outside the safety zone, on the displays. The processor may display views of the virtual space with a designated transparency on the displays. For example, while providing a virtual space defined beyond a safety zone set for the safety of a user wearing the wearable device, the processor may increase the transparency of the virtual space provided using the displays based on identifying a wearable device moving outside the safety zone. By increasing the transparency of the virtual space, the processor may provide the video acquired using at least one camera using the displays. For example, as shown in screen (183) of FIG. 1, the processor may visualize views of the virtual space and the external space substantially simultaneously on the displays.

[0144] Within the operation (860), the processor may display a virtual object to determine whether to stop providing the virtual space based on identifying a wearable device moving outside the safety zone while providing the virtual space defined beyond the safety zone. The virtual object is exemplarily described with reference to FIG. 10A and / or FIG. 10B.

[0145] As described above, according to one embodiment, the processor of the wearable device may conditionally stop providing the virtual space when the wearable device moves outside the safety zone. For example, if the wearable device moves outside the safety zone through a first type of boundary that has a relatively high possibility of colliding with an external object, the processor may immediately stop providing the virtual space. For example, if the wearable device moves outside the safety zone through a second type of boundary that has a relatively low possibility of colliding with an external object, the processor may provide or display a video of the external environment and a video of the virtual space, blended according to transparency. For example, the processor may maintain VR mode or switch to pass-through mode depending on the type of safety zone boundary that the wearable device passes through.

[0146] Hereinafter, exemplary screens displayed by a wearable device performing the operation of FIG. 8 are described with reference to FIGS. 9a, 9b, 9c, 10a and / or 10b.

[0147] FIGS. 9A, 9B, and 9C illustrate exemplary operations of a wearable device (101) for detecting movement of a user (110) wearing the wearable device (101). The wearable device (101) of FIGS. 1 and / or 2, and / or the processor (210) of FIG. 2, may perform operations of the wearable device (101) described with reference to FIGS. 9A to 9C. Operations of the wearable device (101) of FIGS. 9A to 9C may be performed based on the wearable device (101) executing the program of FIG. 2 (e.g., the pass-through manager (253)). Operations of the wearable device (101) described with reference to FIGS. 9A to 9C may be related to at least one of the operations of FIGS. 3, 5, and / or 8.

[0148] Referring to FIG. 9A, an exemplary state (901) of a wearable device (101) positioned at a location (p1) inside a safety zone (910) is illustrated. Referring to FIG. 9A, the safety zone (910) is illustrated in the shape of a rectangle having four vertices (rp1, rp2, rp3, rp4). The shape of the safety zone (910) is not limited thereto. A virtual space (920) provided using displays of the wearable device (101) may be defined by a software application executed by the wearable device (101), independently of the shape and / or size of the safety zone (910).

[0149] Within the exemplary state (901) of FIG. 9A, the wearable device (101) can determine a position within the virtual space corresponding to the current position (p1) of the wearable device (101) by using a mapping between a first coordinate system used to identify the position of the wearable device (101) within the safety zone (910) and a second coordinate system of the virtual space (920). The wearable device (101) can display a view of the virtual space (920) from the determined position within the virtual space by using the displays. The virtual space (920) provided by the wearable device (101) within the state (901) can be displayed through the entire display area of ​​the displays. A user (110) viewing (or recognizing) the virtual space (920) through the displays can move according to the perspective of the virtual space (920).

[0150] Referring to FIG. 9B, an exemplary state (902) is illustrated in which a user (110) has moved to a location (p2). Within the state (902), based on detecting a wearable device (101) positioned at a location (p2) adjacent to the boundary of a safety zone (910), the wearable device (101) may execute a function for notifying the boundary. For example, the wearable device (101) may perform the operations described above with reference to FIGS. 5 to 6 and FIGS. 7A to 7D to display a virtual object notifying the boundary of the safety zone (910) adjacent to the location (p2). For example, at a location (p2) adjacent to a boundary of a second type, the wearable device (101) may display a UI notifying the boundary of the second type, such as the virtual plane (731) of FIG. 7C.

[0151] For example, when a user (110) moves from a location (p2) to a location (p3) outside the safety zone (910), the wearable device (101) may perform the operation of FIG. 8 to determine whether to stop providing the virtual space (920). Referring to the mapping between the safety zone (910) and the virtual space (920) of FIG. 9b, since the location (p3) outside the safety zone (910) corresponds to the outside of the virtual space (920), the virtual space (920) may not be defined at the location (p3). Because the virtual space (920) is not defined at a location (p3) outside the safety zone (910), the virtual space (920) is not set to include the location (p3), or the virtual space (920) does not extend to the location (p3), the wearable device (101) may perform the operation (840) of FIG. 8. For example, at the location (p3), the wearable device (101) may at least temporarily stop providing the virtual space (920). An exemplary screen displayed by the wearable device (101) at the location (p3) of FIG. 9b is described with reference to FIG. 10a.

[0152] Referring to FIG. 9C, an exemplary state (903) is illustrated in which a user (110) has moved to a location (p4). Within the state (903), based on detecting a wearable device (101) positioned at a location (p4) adjacent to the boundary of a safety zone (910), the wearable device (101) may display a virtual object indicating the boundary. For example, at a location (p4) adjacent to a boundary of a second type, the wearable device (101) may display a UI indicating the boundary of the second type, such as the virtual plane (731) of FIG. 7C.

[0153] For example, when a user (110) moves from a location (p4) to a location (p5) outside the safety zone (910), the wearable device (101) may perform the operation of FIG. 8. Referring to FIG. 9C, the location (p5) may be included inside a virtual space (920). For example, the virtual space (920) may be defined at the location (p5). For example, the virtual space (920) may extend from the inside of the safety zone (910) to the location (p5) outside the safety zone (910). Because the virtual space (920) is defined at a location (p5) outside the safety zone (910), or the virtual space (920) is set to include the location (p5), or the virtual space (920) extends to the location (p5), the wearable device (101) can perform the operation (860) of FIG. 8. For example, at the location (p5), the wearable device (101) can provide the virtual space (920). While providing the virtual space (920), the wearable device (101) can display images and / or videos acquired from a camera based on a pass-through mode. An exemplary screen displayed by the wearable device (101) at the location (p5) of FIG. 9c is described with reference to FIG. 10b.

[0154] Hereinafter, with reference to FIGS. 10A and 10B, screens displayed by the wearable device (101) at exemplary locations (p2, p3, p4, p5) of FIGS. 9B and 9C are exemplarily described.

[0155] FIGS. 10A and 10B illustrate exemplary screens (1001, 1002, 1003, 1004) displayed by a wearable device according to a geographic location. The wearable device (101) of FIGS. 1 and / or 2, and / or the processor (210) of FIG. 2, may perform the operations of the wearable device (101) described with reference to FIGS. 10A and 10B. The operations of the wearable device (101) of FIGS. 10A and 10B may be performed by the wearable device (101) executing the program of FIG. 2 (e.g., the pass-through manager (253)). The operation of the wearable device (101) described with reference to FIGS. 10A and 10B may be related to at least one of the operations of FIGS. 3, 5, and / or 8.

[0156] Referring to FIG. 10A, exemplary screens (1001, 1002) displayed by a wearable device are illustrated within an exemplary state (902) of FIG. 9B. The screens (1001, 1002) may be displayed on at least one of the displays configured to be positioned toward the eyes of a user wearing the wearable device (e.g., the user (110) of FIG. 1 ). Referring to FIG. 10A, an exemplary screen (1001) displayed by the wearable device is illustrated while positioned at a location (p2) adjacent to the boundary of a safety zone (910). Since a location within the virtual space (920) corresponding to the location (p2) is adjacent to the boundary of the virtual space (920), a virtual plane representing the boundary of the virtual space (920) may be included within the screen (1001).

[0157] Referring to an exemplary screen (1001) of FIG. 10A, since the location (p2) is adjacent to the boundary of the safety zone (910) (e.g., the boundary line connecting the vertices (rp1, rp2)), the wearable device may display a virtual object (1011) to indicate the boundary. As described above with reference to FIG. 9B, the wearable device that has detected the location (p2) adjacent to the boundary of the second type may display a virtual object (1011) to indicate the boundary of the second type within the screen (1001). A virtual object (1011) having a circular shape is illustrated as an example, but the embodiment is not limited thereto.

[0158] Referring to FIG. 10A, when a user wearing a wearable device moves from a location (p2) to a location (p3), the wearable device may display a screen (1002) on the displays. As described above with reference to FIG. 9B, since the virtual space (920) is not defined at a location (p3) outside the safety zone (910), the wearable device may stop providing the virtual space. The wearable device may display images and / or videos being acquired from a camera (e.g., the camera (225) and / or the external camera (225-2) of FIG. 2) within the screen (1002). For example, a visual object (1021) corresponding to an external object (e.g., the sofa (121) of FIG. 9B) located in the external environment may be displayed through the screen (1002).

[0159] Referring to FIG. 10B , exemplary screens (1003, 1004) displayed by a wearable device are illustrated within the exemplary state (903) of FIG. 9C . The screens (1003, 1004) may be displayed on at least one of the displays included in the wearable device. Referring to FIG. 10B , based on detecting a location (p4) of the wearable device adjacent to the boundary of a safety zone (910), the wearable device may display a screen (1003) including a UI (e.g., a virtual object (1031)) for indicating the boundary. The screen (1003) may include a view of the virtual space (920) as seen from a location in the virtual space (920) corresponding to the location (p4). Within the screen (1003), the virtual object (1031) may have the form of a plane representing the boundary of a safety zone (910) adjacent to the location (p4) (e.g., a boundary line connecting vertices (rp2, rp3)).

[0160] Referring to FIG. 10b, when a user wearing a wearable device moves from location (p4) to location (p5), the wearable device can display a screen (1004) on the displays. As described above with reference to FIG. 9c, since the virtual space (920) is defined at location (p5) outside the safety zone (910), the wearable device can continue to provide the virtual space. For example, within the screen (1004), an image and / or video of the virtual space (920) may be included, expressed based on the location of the virtual space (920) corresponding to location (p5).

[0161] At a location (p5) outside the safety zone (910), in order to provide the user with information about the external environment, the wearable device may display images and / or videos of the external environment within the screen (1004). Referring to FIG. 9C, since the images and / or videos of the external environment are included within the screen (1004), a visual object (1041) corresponding to an external object (e.g., a dog (124)) adjacent to the location (p5) may be displayed on the screen (1004). The images and / or videos of the external environment may be superimposed on images and / or videos of the virtual space (920).

[0162] Referring to FIG. 10B, within the screen (1004), the wearable device may display a virtual object (1042) to confirm whether to stop providing the virtual space (920). The virtual object (1042) in the form of a pop-up window is illustrated as an example, but the embodiment is not limited thereto. The wearable device may display the virtual object (1042) including a designated text to confirm whether to stop the software application running to provide the virtual space (920), such as “Would you like to pause the VR app?” The virtual object (1042) may include virtual objects (1043, 1044) for receiving input related to providing the virtual space (920). While displaying the virtual object (1042), the wearable device may receive input related to at least one of the virtual objects (1043, 1044). The above input may be detected or received based on a gaze, hand gesture, speech, and / or touch gesture of a user wearing the wearable device.

[0163] For example, in response to a first input indicating selection of a virtual object (1043) including designated text, such as “yes,” the wearable device may continue to provide the virtual space (920), such as on the screen (1004). In response to a second input indicating selection of a virtual object (1044) including designated text, such as “no,” the wearable device may stop providing the virtual space (920) using the displays. The wearable device, having received the first input and / or the second input, may at least temporarily stop displaying the virtual object (1042) within the screen (1004).

[0164] As described above, according to one embodiment, the wearable device may determine whether to provide the virtual space (920) outside the safety zone (910) based on the shape and / or state of the virtual space (920) with respect to the geographic location. For example, outside the safety zone (910), the wearable device may continue to provide the virtual space (920). Hereinafter, exemplary operations of the wearable device performed to change the virtual space (920) at the boundary of the safety zone (910) are described with reference to FIGS. 11, 12A, 12B, 12C, 13A, 13B, and 13C.

[0165] FIG. 11 illustrates a flowchart of a wearable device according to one embodiment. The wearable device (101) of FIG. 1 and / or FIG. 2, and / or the processor (210) of FIG. 2, may perform operations of the wearable device described with reference to FIG. 11. The operations of FIG. 11 may be performed by the wearable device (101) executing the program of FIG. 2 (e.g., the pass-through manager (253)). The order in which the operations of FIG. 11 are performed is not limited to the order illustrated in FIG. 11. For example, the processor of the wearable device may perform the operations of FIG. 11 in a different order than the order illustrated in FIG. 11. For example, the processor of the wearable device may perform at least two of the operations of FIG. 11 substantially simultaneously. The operations of FIG. 11 may be at least partially related to the operations of FIG. 3, FIG. 5, and / or FIG. 8.

[0166] Referring to FIG. 11, in operation (1110), a processor of a wearable device according to one embodiment may provide a virtual space using displays. The processor may perform operation (1110) similar to operation (510) of FIG. 5 and / or operation (810) of FIG. 8. For example, the processor may display an image and / or video representing at least a portion of the virtual space to a user wearing the wearable device (e.g., user (110) of FIG. 1). While providing the virtual space in operation (1110), the processor may repeatedly perform operation (1120).

[0167] Referring to FIG. 11, in operation 1120, a processor of a wearable device according to one embodiment may determine or identify whether a distance between a first type boundary of a safety zone and the wearable device and / or a designated body part is less than a threshold distance. The safety zone may be established based on the operations described with reference to FIGS. 3 and 4. Based on a distance between an external object located outside the safety zone and the boundary of the safety zone, the processor may classify at least a portion of the boundary as a first type boundary. For example, the first type boundary may be established based on detecting an external object that is less than a designated distance from the boundary. The second type boundary may be established when the closest external object to the boundary is more than a designated distance from the boundary. The processor can perform operation (1120) of FIG. 11 similarly to operation (520) of FIG. 5.

[0168] If the distance between the boundary of the first type of the safety zone and the wearable device and / or the designated body part is greater than or equal to the threshold distance (1120-No), the processor may continue to provide the virtual space of the operation (1110). If the distance between the boundary of the first type of the safety zone and the wearable device and / or the designated body part is less than the threshold distance (1120-Yes), the processor may perform the operation (1130).

[0169] Referring to FIG. 11, in operation (1130), a processor of a wearable device according to one embodiment may execute a function for notifying an approach to a first type of boundary. Operation (1130) may be performed similarly to operation (530) of FIG. 5. The processor may display a virtual object representing a first type of boundary on displays, such as the virtual plane (721) of FIG. 7B.

[0170] Referring to FIG. 11, within operation (1140), a processor of a wearable device according to one embodiment may determine or identify whether a virtual space is defined in an area (or a location of the wearable device) beyond a first type of boundary. For example, the processor may perform operation (1140) similar to operation (850) of FIG. 8. If the virtual space is not defined at a location beyond the first type of boundary, or if the virtual space does not extend to a location beyond the first type of boundary, or if the virtual space does not include a location beyond the first type of boundary (1140-No), the processor may continue performing at least one of operations (1110, 1120).

[0171] For example, if the virtual space is defined at a location beyond a boundary of the first type, or if the virtual space extends to a location beyond a boundary of the first type, or if the virtual space includes a location beyond a boundary of the first type (1140-Yes), the processor may perform operation (1150).

[0172] Referring to FIG. 11, in operation 1150, a processor of a wearable device according to one embodiment may execute a function for changing the linkage between a safety zone and a virtual space. For example, the processor may execute a function for changing or rotating the virtual space and / or a coordinate system of the virtual space without movement of the wearable device (or independently of movement of the wearable device). The function may include an operation of displaying a virtual object to determine whether to at least partially change the virtual space. The function may be initiated by performing the operation.

[0173] For example, while providing a first portion of a virtual space linked to a safety zone using displays, the processor can perform operation (1140) to identify a second portion of the virtual space linked to an external space beyond the first type of boundary. The second portion can be connected to the first portion, or can be located next to the first portion within a coordinate system of the virtual space. The processor can provide or display a virtual object using displays to identify whether to provide the second portion defined beyond the first type of boundary using displays.

[0174] As described above, in a position adjacent to a first type of boundary adjacent to an external object, such as an obstacle, the wearable device may support interaction with a virtual space (or virtual object) associated with the boundary beyond the first type. To support the interaction, the wearable device may change the relationship (or mapping) between the safety zone and the virtual space such that at least a portion of the virtual space that may be provided using the displays while the wearable device is positioned inside the safety zone includes another portion of the virtual object associated with the boundary beyond the first type. The function of operation (1150) may be referred to as reorientation and / or realignment.

[0175] Below, exemplary screens displayed by a wearable device performing the operations of FIG. 11 are described with reference to FIGS. 12a to 12c and FIGS. 13a to 13c.

[0176] FIGS. 12A, 12B, and 12C illustrate exemplary operations of a wearable device (101) that change the relationship between a geographic location of the wearable device (101) and a virtual space (920). The wearable device (101) of FIGS. 1 and / or 2, and / or the processor (210) of FIG. 2, may perform the operations of the wearable device (101) described with reference to FIGS. 12A to 12C. The operations of the wearable device (101) of FIGS. 12A to 12C may be performed based on the execution of the program of FIG. 2 (e.g., the pass-through manager (253)). The operation of the wearable device (101) described with reference to FIGS. 12a to 12c may be related to at least one of the operations of FIGS. 3, 5, 8, and / or 11.

[0177] Referring to FIG. 12A, an exemplary state (1201) of a wearable device (101) positioned at a location (p1) inside a safety zone (910) is illustrated. Within the state (1201), a user (110) wearing the wearable device (101) may move to a location (p6). The wearable device (101) may use a sensor (e.g., sensor (230) of FIG. 2) to identify or detect the location of the wearable device (101) and / or the user (110) that has moved to the location (p6). In an exemplary state (1201) where the bookshelf (123) is adjacent to a portion of the boundary of the safety zone (910) (e.g., a portion of the boundary connecting the vertices (rp1, rp4)), the portion of the boundary adjacent to the position (p6) can be determined as a boundary of the first type.

[0178] Within the exemplary state (1201) of FIG. 12A, the wearable device (101) can visualize a virtual space based on the position (p6) and orientation (d1) of the wearable device (101). Referring to FIG. 12B, within the state (1201) of FIG. 12A, an exemplary screen (1220) displayed on the displays is illustrated. In one embodiment, where a portion (912) of the virtual space (920) is defined beyond the safety zone (910) adjacent to the position (p6), the wearable device (101) can display an image and / or video representing the portion (912) of the virtual space (920) on the screen (1220). Referring to FIG. 12b, at a location (p6) adjacent to the boundary of a safety zone (910), a user (110) wearing a wearable device (101) can see a virtual object (1221) indicating a boundary of the first type, together with a portion (912) in a virtual space (920). The virtual object (1221) may correspond to the virtual plane (721) of FIG. 7b.

[0179] In the exemplary case of FIG. 12b, a user (110) viewing a portion (912) displayed on a display may move outside the safety zone (910) to move to the portion (912). When moving along the direction (d1), the user (110) viewing the portion (912) may collide with an external object, such as a bookshelf (123). To prevent the collision while supporting movement to the portion (912), the wearable device (101) may support a function for changing the mapping of the safety zone (910) and the virtual space (920). For example, while providing a virtual space (920) defined beyond a safety zone (910), based on identifying a wearable device (101) moving toward the boundary of the safety zone (910) adjacent to an external object (e.g., a bookshelf (123)), the wearable device (101) may provide a virtual object (150) using displays to change the mapping between the safety zone (910) and the virtual space (920) (e.g., the placement location and / or placement direction of the virtual space (920) with respect to the safety zone (910).

[0180] While displaying the screen (1220) of FIG. 12B, the wearable device (101) may receive an input related to the virtual object (150). The input may be detected or identified based on a signal received from an external electronic device (e.g., a remote controller (410) of FIG. 4) connected to the wearable device (101). The input may be detected based on gaze information of a user wearing the wearable device (101) (e.g., a gaze position that appears to be directed toward the virtual object (150). The input may be detected based on a hand gesture of a user wearing the wearable device (101). The input may be detected based on a speech of a user wearing the wearable device (101).

[0181] In response to an input related to a virtual object (150), the wearable device (101) can change the mapping between the safety zone (910) and the virtual space (920). For example, the wearable device (101) can change the mapping such that a portion (912) of the virtual space (920) defined beyond the safety zone (910) moves into the safety zone (910). Referring to FIG. 12C, an exemplary state of the wearable device (101) after changing the mapping between the safety zone (910) and the virtual space (920) in response to an input related to the virtual object (150) is illustrated. For example, the wearable device (101) can rotate the virtual space (920) (or the coordinate system of the virtual space (920)) based on the current position (p6) of the wearable device (101). For example, the wearable device (101) may rotate the virtual space (920) so that a portion (921) of the virtual space (920) that was positioned in front of the wearable device (101) (e.g., in the direction (d1)) is positioned in a portion (e.g., a portion including the vertex (rp2)) of the safety zone (910) that is furthest from the location (p6).

[0182] To rotate the virtual space (920), the wearable device (101) can calculate or identify the direction (d2) of a portion farthest from the position (p6) of the wearable device (101) within the safety zone (910). Having identified the direction (d2), the wearable device (101) can rotate the virtual space (920) so that a portion (921) of the virtual space (920) located in the direction (d1) is located in the direction (d2). Referring to FIG. 12C, based on the rotation, the portion (921) can be located in a portion of the safety zone (910) that is farthest from the position (p6).

[0183] Although exemplary operations of a wearable device (101) rotating a virtual space (920) have been described, the embodiments are not limited thereto. The wearable device (101) can change the mapping of the safety zone (910) and the virtual space (920) by executing a computational model based on artificial intelligence. For example, by executing the computational model, the wearable device (101) can determine or calculate the rotation angle of the virtual space (920). The wearable device (101) can change the mapping of the safety zone (910) and the virtual space (920) by utilizing the shape of a movable part within the virtual space (920).

[0184] Referring to FIG. 12C, a screen (1230) displayed by a wearable device (101) that has changed the coordinate system of a virtual space (920) based on a safety zone (910) is illustrated. After changing the mapping between the safety zone (910) and the virtual space (920) in response to an input related to a virtual object (150), the wearable device (101) may display, on its displays, a virtual object (1222) indicating the direction of a portion (921) of the virtual space (920) that was defined beyond the boundary before changing the mapping. Referring to FIG. 12C, within the screen (1230), the wearable device (101) may display, within the screen (1230), a virtual object (1222) including text (e.g., "TURN LEFT") indicating the direction (d2) of the portion (921). At a location (p6) adjacent to the boundary of the safety zone, the wearable device (101) can continuously display a virtual object (1221) representing the boundary.

[0185] Referring to FIG. 12c, a screen (1240) displayed by the wearable device (101) is illustrated after a user (110) wearing the wearable device (101) rotates toward a direction (d2). For example, a user (110) who sees a virtual object (1222) in the screen (1230) may rotate toward a direction (d2). The wearable device (101) may display the screen (1240) on the displays based on detecting the direction (d2) of the wearable device (101) changed by the rotation. Since the wearable device (101) is not facing the boundary of the safety zone (910), a virtual object (e.g., virtual object (1221)) representing the boundary may not be displayed in the screen (1240). For example, before moving to the boundary of the safety zone (910), the user (110) may move to a location inside the safety zone (910), corresponding to a portion (921) of the virtual space (920) along the direction (d2).

[0186] As described above, according to one embodiment, the wearable device (101) may support access beyond a portion of the virtual space (920) mapped to the safety zone (910) when providing a virtual space (920) having a size larger than the safety zone (910). At a location (p6) adjacent to the boundary of the safety zone (910), the wearable device (101) may display a virtual object (150) for accessing a portion (921) of the virtual space (920) defined beyond the boundary. In response to an input related to the virtual object (150), the wearable device (101) may rotate or rearrange the virtual space (920). Based on the above rotation and / or the above rearrangement, within a safety zone (910) having a size smaller than the virtual space (920), the wearable device (101) can support accessibility to the entire virtual space (920).

[0187] Hereinafter, with reference to FIGS. 13a to 13c, exemplary operations of a wearable device (101) for supporting accessibility within a virtual space (920) wider than a safety zone (910) are described.

[0188] FIGS. 13A, 13B, and 13C illustrate exemplary operations of a wearable device (101) that change a relationship between a geographic location of the wearable device (101) and a virtual object (e.g., virtual objects (1311, 1312, 1313)). The wearable device (101) of FIGS. 1 and / or 2, and / or the processor (210) of FIG. 2, may perform the operations of the wearable device (101) described with reference to FIGS. 13A to 13C. The operations of the wearable device (101) of FIGS. 13A to 13C may be performed based on the execution of the program (e.g., the pass-through manager (253)) of FIG. 2. The operation of the wearable device (101) described with reference to FIGS. 13a to 13c may be related to at least one of the operations of FIGS. 3, 5, 8, and / or 11.

[0189] Referring to FIG. 13A, an exemplary state (1301) of a wearable device (101) positioned at a location (p1) inside a safety zone (910) is illustrated. Within the state (1301), the wearable device (101) may provide virtual objects (1311, 1312, 1313) that are respectively linked to locations (v1, v2, v3) of the external environment. For example, on displays, the wearable device (101) may display a screen (1391) based on the location (p1) and direction (d1) of the wearable device (101). Referring to FIG. 13a, virtual objects (1311, 1312, 1313) can be displayed at respective locations (v1, v2, v3) within the screen (1391) viewed from location (v1) along direction (d1).

[0190] For example, the size of the virtual space managed by the wearable device (101) may be different from the size of the safety zone (910). Referring to FIG. 13a, a virtual object (1311) may be linked to a location (v1) outside the safety zone (910). Referring to FIG. 13b, an exemplary state (1302) is illustrated in which a user (110) moves toward the virtual object (1311) to interact with the virtual object (1311). Within the state (1302), when the user (110) moves to a location (p2) of the safety zone (910) adjacent to the virtual object (1311), the wearable device (101) may execute a function to notify the boundary of the safety zone (910). Since the boundary of the adjacent safety zone (910) from the location (p2) is a first type boundary adjacent to an external object such as a bookshelf (123), the wearable device (101) can display a screen (1392) including a virtual object (1321) for indicating the first type boundary. The virtual object (1321) can correspond to the virtual plane (721) of FIG. 7B.

[0191] Referring to FIG. 13b, at a position (p2) adjacent to the boundary of the safety zone (910), the wearable device (101) can display a screen (1392) including a virtual object (1321) expressing the boundary to a user (110) facing in a direction (d2). The wearable device (101) can identify the virtual object (1311) having a position (v1) outside the boundary of the safety zone (910). The wearable device (101) displaying the screen (1392) including the virtual object (1311) can display a virtual object (150) configured to receive an input for moving the virtual object (1311) toward the inside of the safety zone (910).

[0192] In response to an input related to a virtual object (150), the wearable device (101) can change the mapping between the safety zone (910) and the virtual space. The wearable device (101) can change the location (v1) of at least one virtual object (e.g., virtual object (1311)) located in the safety zone (910) and the virtual space. For example, the wearable device (101) can change the coordinates of the virtual object (1311) in order to move the virtual object (1311) located outside the safety zone (910) to inside the safety zone (910).

[0193] Referring to FIG. 13c, an exemplary state (1303) is illustrated in which the positions (v1, v2, v3) of virtual objects (1311, 1312, 1313) are changed in response to an input related to a virtual object (150). Within the state (1303) of FIG. 13c, the wearable device (101) may rotate the virtual objects (1311, 1312, 1313) based on the current position (p2) of the wearable device (101). Based on the rotation, the virtual object (1311) positioned outside the safety zone (910) may move inside the safety zone (910). After moving the virtual object (1311), the wearable device (101) can display a virtual object (1331) including text (e.g., “TURN BACK”) indicating the position and / or direction of the virtual object (1311) within the screen (1393) so that the user (110) viewing the direction (d2) can turn toward the virtual object (1311).

[0194] For example, the screen (1393) may include a virtual object (1331) for guiding the location of the virtual object (1311), together with a virtual object (1321) for indicating a boundary of the first type, a virtual space provided by the wearable device (101). A user (110) viewing the virtual object (1331) may rotate toward a direction (d3) inside the safety zone (910). The user (110) that rotates toward the direction (d3) may view the virtual object (1311) again. Since the virtual object (1311) is located inside the safety zone (910), the user (110) may move toward the virtual object (1311) again. The wearable device (101) can support interaction between a virtual object (1311) and a user (110) inside the safety zone (910).

[0195] As described above, in order to enhance accessibility to a virtual space defined more broadly than the safety zone (910), the wearable device (101) according to one embodiment may provide a UI related to the virtual space at the boundary of the safety zone (910) or outside the safety zone (910). For example, the wearable device (101) may conditionally continue to provide the virtual space even after moving outside the safety zone (910). For example, the wearable device (101) may change (e.g., rotate) the coordinate system of the virtual space at a location adjacent to the boundary of the safety zone (910), thereby allowing the user to change a portion of the virtual space mapped to the safety zone (910).

[0196] Hereinafter, with reference to FIGS. 14A, 14B, 15A, and 15B, an exemplary structure of a wearable device (101) of FIG. 1 and / or FIG. 2 is illustrated. The wearable device (1400) of FIGS. 14A to 14B and the wearable device (1500) of FIGS. 15A to 15B may be an example of the wearable device (101) of FIGS. 1 and / or 2.

[0197] FIG. 14A illustrates an example of a perspective view of a wearable device, according to one embodiment. According to one embodiment, the wearable device (1400) may have the form of glasses that are wearable on a body part of a user (e.g., head). The wearable device (1400) may include a head-mounted display (HMD). For example, the housing of the wearable device (1400) may include a flexible material, such as rubber and / or silicone, that is configured to fit closely to a portion of the user's head (e.g., a portion of the face surrounding both eyes). For example, the housing of the wearable device (1400) may include one or more straps that are capable of being twined around the user's head, and / or one or more temples that are detachably attachable to the ears of the head.

[0198] Referring to FIG. 14A, according to one embodiment, a wearable device (1400) may include at least one display (1450) and a frame supporting at least one display (1450).

[0199] According to one embodiment, a wearable device (1400) can be worn on a part of a user's body. The wearable device (1400) can provide augmented reality (AR), virtual reality (VR), or mixed reality (MR) that combines augmented reality and virtual reality to a user wearing the wearable device (1400). For example, the wearable device (1400) can display a virtual reality image provided from at least one optical device (1482, 1484) of FIG. 14B on at least one display (1450) in response to a user's designated gesture acquired through the motion recognition cameras (1460-2, 1460-3) of FIG. 14B.

[0200] According to one embodiment, at least one display (1450) may provide visual information to a user. For example, at least one display (1450) may include a transparent or translucent lens. At least one display (1450) may include a first display (1450-1) and / or a second display (1450-2) spaced apart from the first display (1450-1). For example, the first display (1450-1) and the second display (1450-2) may be positioned at positions corresponding to the user's left and right eyes, respectively.

[0201] Referring to FIG. 14B, at least one display (1450) can provide visual information transmitted from external light to a user through a lens included in the at least one display (1450), and other visual information distinct from the visual information. The lens can be formed based on at least one of a Fresnel lens, a pancake lens, or a multi-channel lens. For example, at least one display (1450) can include a first surface (1431) and a second surface (1432) opposite to the first surface (1431). A display area can be formed on the second surface (1432) of the at least one display (1450). When a user wears the wearable device (1400), external light can be transmitted to the user by being incident on the first surface (1431) and transmitted through the second surface (1432). As another example, at least one display (1450) can display an augmented reality image combined with a virtual reality image provided from at least one optical device (1482, 1484) on a real screen transmitted through external light, in a display area formed on the second surface (1432).

[0202] In one embodiment, at least one display (1450) may include at least one waveguide (1433, 1434) that diffracts light emitted from at least one optical device (1482, 1484) and transmits the diffracted light to a user. The at least one waveguide (1433, 1434) may be formed based on at least one of glass, plastic, or polymer. A nano-pattern may be formed on at least a portion of the exterior or interior of the at least one waveguide (1433, 1434). The nano-pattern may be formed based on a grating structure having a polygonal and / or curved shape. Light incident on one end of the at least one waveguide (1433, 1434) may be propagated to the other end of the at least one waveguide (1433, 1434) by the nano-pattern. At least one waveguide (1433, 1434) may include at least one diffractive element (e.g., a diffractive optical element (DOE), a holographic optical element (HOE)) and at least one reflective element (e.g., a reflective mirror). For example, at least one waveguide (1433, 1434) may be arranged within the wearable device (1400) to guide a screen displayed by at least one display (1450) to the user's eyes. For example, the screen may be transmitted to the user's eyes based on total internal reflection (TIR) ​​occurring within the at least one waveguide (1433, 1434).

[0203] The wearable device (1400) can analyze an object included in a real image collected through a shooting camera (1460-4), combine a virtual object corresponding to an object to be provided with augmented reality among the analyzed objects, and display the virtual object on at least one display (1450). The virtual object can include at least one of text and an image regarding various information related to the object included in the real image. The wearable device (1400) can analyze the object based on a multi-camera such as a stereo camera. For the object analysis, the wearable device (1400) can perform spatial recognition (e.g., simultaneous localization and mapping (SLAM)) using the multi-camera and / or time-of-flight (ToF). A user wearing the wearable device (1400) can view an image displayed on at least one display (1450).

[0204] According to one embodiment, the frame may be formed as a physical structure that allows the wearable device (1400) to be worn on the user's body. According to one embodiment, the frame may be configured so that, when the user wears the wearable device (1400), the first display (1450-1) and the second display (1450-2) can be positioned corresponding to the user's left and right eyes. The frame may support at least one display (1450). For example, the frame may support the first display (1450-1) and the second display (1450-2) to be positioned corresponding to the user's left and right eyes.

[0205] Referring to FIG. 14A, the frame may include a region (1420) that at least partially contacts a portion of the user's body when the user wears the wearable device (1400). For example, the region (1420) of the frame that contacts a portion of the user's body may include a region that contacts a portion of the user's nose, a portion of the user's ear, and a portion of the side of the user's face that the wearable device (1400) makes contact with. According to one embodiment, the frame may include a nose pad (1410) that contacts a portion of the user's body. When the wearable device (1400) is worn by the user, the nose pad (1410) may contact a portion of the user's nose. The frame may include a first temple (1404) and a second temple (1405) that contact a portion of the user's body that is distinct from the portion of the user's body.

[0206] For example, the frame may include a first rim (1401) that surrounds at least a portion of a first display (1450-1), a second rim (1402) that surrounds at least a portion of a second display (1450-2), a bridge (1403) that is disposed between the first rim (1401) and the second rim (1402), a first pad (1411) that is disposed along a portion of an edge of the first rim (1401) from one end of the bridge (1403), a second pad (1412) that is disposed along a portion of an edge of the second rim (1402) from the other end of the bridge (1403), a first temple (1404) that extends from the first rim (1401) and is secured to a portion of an ear of the wearer, and a second temple (1405) that extends from the second rim (1402) and is secured to a portion of an ear opposite the ear. The first pad (1411) and the second pad (1412) may be in contact with a portion of the user's nose, and the first temple (1404) and the second temple (1405) may be in contact with a portion of the user's face and a portion of the user's ear. The temples (1404, 1405) may be rotatably connected to the rim through the hinge units (1406, 1407) of FIG. 14B. The first temple (1404) may be rotatably connected to the first rim (1401) through the first hinge unit (1406) disposed between the first rim (1401) and the first temple (1404). The second temple (1405) may be rotatably connected to the second rim (1402) via a second hinge unit (1407) disposed between the second rim (1402) and the second temple (1405). In one embodiment, the wearable device (1400) may use a touch sensor, a grip sensor, and / or a proximity sensor formed on at least a portion of a surface of the frame to identify an external object (e.g., a user's fingertip) touching the frame and / or a gesture performed by the external object.

[0207] According to one embodiment, the wearable device (1400) may include hardwares (e.g., hardwares described above based on the block diagram of FIG. 2) that perform various functions. For example, the hardwares may include a battery module (1470), an antenna module (1475), at least one optical device (1482, 1484), speakers (e.g., speakers 1455-1, 1455-2), a microphone (e.g., microphones 1465-1, 1465-2, 1465-3), a light-emitting module, and / or a printed circuit board (PCB) (1490) (e.g., a printed circuit board). The various hardware components may be arranged within a frame.

[0208] According to one embodiment, the microphones (e.g., microphones 1465-1, 1465-2, 1465-3) of the wearable device (1400) may be disposed on at least a portion of the frame to acquire sound signals. A first microphone (1465-1) disposed on the bridge (1403), a second microphone (1465-2) disposed on the second rim (1402), and a third microphone (1465-3) disposed on the first rim (1401) are illustrated in FIG. 14B , but the number and arrangement of the microphones (1465) are not limited to the embodiment of FIG. 14B . When the number of microphones (1465) included in the wearable device (1400) is two or more, the wearable device (1400) may identify the direction of the sound signal by using a plurality of microphones disposed on different portions of the frame.

[0209] According to one embodiment, at least one optical device (1482, 1484) can project a virtual object onto at least one display (1450) to provide various image information to a user. For example, at least one optical device (1482, 1484) can be a projector. At least one optical device (1482, 1484) can be positioned adjacent to at least one display (1450) or can be included within at least one display (1450) as a part of at least one display (1450). According to one embodiment, the wearable device (1400) can include a first optical device (1482) corresponding to a first display (1450-1) and a second optical device (1484) corresponding to a second display (1450-2). For example, at least one optical device (1482, 1484) may include a first optical device (1482) disposed at an edge of a first display (1450-1) and a second optical device (1484) disposed at an edge of a second display (1450-2). The first optical device (1482) may transmit light to a first waveguide (1433) disposed on the first display (1450-1), and the second optical device (1484) may transmit light to a second waveguide (1434) disposed on the second display (1450-2).

[0210] In one embodiment, the camera (1460) may include a recording camera (1460-4), an eye tracking camera (ET CAM) (1460-1), and / or motion recognition cameras (1460-2, 1460-3). The recording camera (1460-4), the eye tracking camera (1460-1), and the motion recognition cameras (1460-2, 1460-3) may be positioned at different locations on the frame and may perform different functions. The eye tracking camera (1460-1) may output data indicating the position or gaze of the eyes of a user wearing the wearable device (1400). For example, the wearable device (1400) may detect the gaze from an image including the user's pupils obtained through the eye tracking camera (1460-1).

[0211] The wearable device (1400) can identify an object (e.g., a real object and / or a virtual object) focused on by the user using the user's gaze acquired through the gaze tracking camera (1460-1). The wearable device (1400) that has identified the focused object can execute a function (e.g., gaze interaction) for interaction between the user and the focused object. The wearable device (1400) can express a part corresponding to the eye of an avatar representing the user in a virtual space using the user's gaze acquired through the gaze tracking camera (1460-1). The wearable device (1400) can render an image (or screen) displayed on at least one display (1450) based on the position of the user's eyes.

[0212] For example, the visual quality of a first region related to the gaze within an image and the visual quality (e.g., resolution, brightness, saturation, grayscale, PPI (pixels per inch)) of a second region distinct from the first region may be different from each other. The wearable device (1400) may obtain an image having the visual quality of the first region and the visual quality of the second region that match the gaze of the user using foveated rendering. For example, if the wearable device (1400) supports an iris recognition function, user authentication may be performed based on iris information obtained using a gaze tracking camera (1460-1). Although an example in which the gaze tracking camera (1460-1) is positioned toward the user's right eye is illustrated in FIG. 14B, the embodiment is not limited thereto, and the gaze tracking camera (1460-1) may be positioned solely toward the user's left eye, or toward both eyes.

[0213] In one embodiment, the capturing camera (1460-4) can capture an actual image or background to be aligned with a virtual image to implement augmented reality or mixed reality content. The capturing camera (1460-4) can be used to obtain a high-resolution image based on HR (high resolution) or PV (photo video). The capturing camera (1460-4) can capture an image of a specific object existing at a location viewed by the user and provide the image to at least one display (1450). The at least one display (1450) can display a single image in which information about an actual image or background including an image of the specific object obtained using the capturing camera (1460-4) and a virtual image provided through at least one optical device (1482, 1484) are superimposed. The wearable device (1400) can compensate for depth information (e.g., the distance between the wearable device (1400) and an external object acquired through a depth sensor) using an image acquired through the capture camera (1460-4). The wearable device (1400) can perform object recognition using an image acquired through the capture camera (1460-4). The wearable device (1400) can perform a function of focusing on an object (or subject) in an image (e.g., auto focus) and / or an optical image stabilization (OIS) function (e.g., anti-shake function) using the capture camera (1460-4). The wearable device (1400) can perform a pass-through function to display an image acquired through the capture camera (1460-4) by overlapping at least a portion of a screen representing a virtual space on at least one display (1450) while displaying a screen. In one embodiment, the capturing camera (1460-4) may be positioned on a bridge (1403) positioned between the first rim (1401) and the second rim (1402).

[0214] The gaze tracking camera (1460-1) can implement more realistic augmented reality by tracking the gaze of a user wearing a wearable device (1400) and matching the user's gaze with visual information provided to at least one display (1450). For example, when the wearable device (1400) looks straight ahead, the wearable device (1400) can naturally display environmental information related to the user's front at a location where the user is located on at least one display (1450). The gaze tracking camera (1460-1) can be configured to capture an image of the user's pupil to determine the user's gaze. For example, the gaze tracking camera (1460-1) can receive gaze detection light reflected from the user's pupil and track the user's gaze based on the position and movement of the received gaze detection light. In one embodiment, the gaze tracking camera (1460-1) can be positioned at positions corresponding to the user's left and right eyes. For example, the gaze tracking camera (1460-1) may be positioned within the first rim (1401) and / or the second rim (1402) to face the direction in which the user wearing the wearable device (1400) is positioned.

[0215] The gesture recognition cameras (1460-2, 1460-3) can recognize the movement of the user's entire body, such as the user's torso, hands, or face, or a part of the body, and thereby provide a specific event on a screen provided on at least one display (1450). The gesture recognition cameras (1460-2, 1460-3) can recognize the user's gesture (gesture recognition), obtain a signal corresponding to the gesture, and provide a display corresponding to the signal on at least one display (1450). The processor can identify the signal corresponding to the gesture, and perform a designated function based on the identification. The gesture recognition cameras (1460-2, 1460-3) can be used to perform a spatial recognition function using SLAM and / or a depth map for 6 degrees of freedom pose (6 dof pose). The processor may perform gesture recognition and / or object tracking functions using the motion recognition cameras (1460-2, 1460-3). In one embodiment, the motion recognition cameras (1460-2, 1460-3) may be positioned on the first rim (1401) and / or the second rim (1402).

[0216] The camera (1460) included in the wearable device (1400) is not limited to the above-described gaze tracking camera (1460-1) and motion recognition cameras (1460-2, 1460-3). For example, the wearable device (1400) can identify an external object included in the FoV using a camera positioned toward the user's FoV. The wearable device (1400) can identify an external object based on a sensor for identifying the distance between the wearable device (1400) and the external object, such as a depth sensor and / or a time of flight (ToF) sensor. The camera (1460) positioned toward the FoV can support an autofocus function and / or an optical image stabilization (OIS) function. For example, the wearable device (1400) may include a camera (1460) (e.g., a face tracking (FT) camera) positioned toward the face to obtain an image including the face of a user wearing the wearable device (1400).

[0217] Although not shown, in one embodiment, the wearable device (1400) may further include a light source (e.g., an LED) that emits light toward a subject (e.g., a user's eyes, face, and / or an external object within the FoV) being captured using the camera (1460). The light source may include an infrared wavelength LED. The light source may be disposed on at least one of the frame and hinge units (1406, 1407).

[0218] In one embodiment, the battery module (1470) may supply power to electronic components of the wearable device (1400). In one embodiment, the battery module (1470) may be disposed within the first temple (1404) and / or the second temple (1405). For example, the battery module (1470) may be a plurality of battery modules (1470). The plurality of battery modules (1470) may be disposed within each of the first temple (1404) and the second temple (1405). In one embodiment, the battery module (1470) may be disposed at an end of the first temple (1404) and / or the second temple (1405).

[0219] The antenna module (1475) can transmit signals or power to the outside of the wearable device (1400), or receive signals or power from the outside. In one embodiment, the antenna module (1475) can be positioned within the first temple (1404) and / or the second temple (1405). For example, the antenna module (1475) can be positioned close to one surface of the first temple (1404) and / or the second temple (1405).

[0220] The speaker (1455) can output an audio signal to the outside of the wearable device (1400). The audio output module may be referred to as a speaker. In one embodiment, the speaker (1455) may be positioned within the first temple (1404) and / or the second temple (1405) so as to be positioned adjacent to the ear of a user wearing the wearable device (1400). For example, the speaker (1455) may include a second speaker (1455-2) positioned within the first temple (1404) and thus adjacent to the user's left ear, and a first speaker (1455-1) positioned within the second temple (1405) and thus adjacent to the user's right ear.

[0221] The light-emitting module (not shown) may include at least one light-emitting element. The light-emitting module may emit light of a color corresponding to a specific state or emit light with an action corresponding to a specific state in order to visually provide information regarding a specific state of the wearable device (1400) to the user. For example, when the wearable device (1400) requires charging, it may emit red light at a regular cycle. In one embodiment, the light-emitting module may be disposed on the first rim (1401) and / or the second rim (1402).

[0222] Referring to FIG. 14B, according to one embodiment, a wearable device (1400) may include a printed circuit board (PCB) (1490). The PCB (1490) may be included in at least one of the first temple (1404) or the second temple (1405). The PCB (1490) may include an interposer disposed between at least two sub-PCBs. One or more hardwares included in the wearable device (1400) (e.g., hardwares illustrated by different blocks in FIG. 2) may be disposed on the PCB (1490). The wearable device (1400) may include a flexible PCB (FPCB) for interconnecting the hardwares.

[0223] According to one embodiment, a wearable device (1400) may include at least one of a gyro sensor, a gravity sensor, and / or an acceleration sensor for detecting a posture of the wearable device (1400) and / or a posture of a body part (e.g., a head) of a user wearing the wearable device (1400). Each of the gravity sensor and the acceleration sensor may measure gravitational acceleration and / or acceleration based on mutually perpendicular designated three-dimensional axes (e.g., an x-axis, a y-axis, and a z-axis). The gyro sensor may measure an angular velocity of each of the designated three-dimensional axes (e.g., an x-axis, a y-axis, and a z-axis). At least one of the gravity sensor, the acceleration sensor, and the gyro sensor may be referred to as an inertial measurement unit (IMU). According to one embodiment, the wearable device (1400) may identify a user's motion and / or gesture performed to execute or terminate a specific function of the wearable device (1400) based on the IMU.

[0224] Figures 15A and 15B illustrate an example of an exterior appearance of a wearable device according to one embodiment. The wearable device (1500) of Figures 15A and 15B may include at least a portion of the hardware of the wearable device (1400) described with reference to Figures 14A and / or 14B. An example of an exterior appearance of a first side (1510) of a housing of the wearable device (1500) according to one embodiment is illustrated in Figure 15A, and an example of an exterior appearance of a second side (1520) opposite to the first side (1510) may be illustrated in Figure 15B.

[0225] Referring to FIG. 15A, a first surface (1510) of a wearable device (1500) according to one embodiment may have a form attachable to a body part of a user (e.g., the face of the user). Although not shown, the wearable device (1500) may further include a strap for fixing to a body part of a user, and / or one or more temples (e.g., the first temple (1404) and / or the second temple (1405) of FIGS. 14A and 14B). A first display (1450-1) for outputting an image to a left eye among the user's two eyes, and a second display (1450-2) for outputting an image to a right eye among the two eyes, may be disposed on the first surface (1510). The wearable device (1500) may be formed on the first surface (1510) and may further include a rubber or silicone packing to prevent interference from light (e.g., ambient light) different from the light emitted from the first display (1450-1) and the second display (1450-2).

[0226] According to one embodiment, a wearable device (1500) may include cameras (1460-1) for photographing and / or tracking both eyes of a user adjacent to each of the first display (1450-1) and the second display (1450-2). The cameras (1460-1) may be referred to as the gaze tracking camera (1460-1) of FIG. 14B. According to one embodiment, a wearable device (1500) may include cameras (1460-5, 1460-6) for photographing and / or recognizing a face of a user. The cameras (1460-5, 1460-6) may be referred to as FT cameras. The wearable device (1500) can control an avatar representing the user in a virtual space based on the facial motion of the user identified using cameras (1460-5, 1460-6). For example, the wearable device (1500) can change the texture and / or shape of a portion of the avatar (e.g., a portion of the avatar representing a human face) using information obtained by cameras (1460-5, 1460-6) (e.g., FT cameras) and representing the facial expression of the user wearing the wearable device (1500).

[0227] Referring to FIG. 15b, a camera (e.g., cameras (1460-7, 1460-8, 1460-9, 1460-10, 1460-11, 1460-12)) and / or a sensor (e.g., a depth sensor (1530)) for obtaining information related to the external environment of the wearable device (1500) may be disposed on a second surface (1520) opposite to the first surface (1510) of FIG. 15a. For example, the cameras (1460-7, 1460-8, 1460-9, 1460-10) may be disposed on the second surface (1520) to recognize external objects. The cameras (1460-7, 1460-8, 1460-9, 1460-10) of FIG. 15b can correspond to the motion recognition cameras (1460-2, 1460-3) of FIG. 14b.

[0228] For example, using cameras (1460-11, 1460-12), the wearable device (1500) can obtain images and / or videos to be transmitted to each of the user's eyes. The camera (1460-11) can be placed on the second face (1520) of the wearable device (1500) to obtain an image to be displayed through the second display (1450-2) corresponding to the right eye among the two eyes. The camera (1460-12) can be placed on the second face (1520) of the wearable device (1500) to obtain an image to be displayed through the first display (1450-1) corresponding to the left eye among the two eyes. The cameras (1460-11, 1460-12) can correspond to the shooting camera (1460-4) of FIG. 14B.

[0229] According to one embodiment, the wearable device (1500) may include a depth sensor (1530) disposed on a second face (1520) to identify a distance between the wearable device (1500) and an external object. Using the depth sensor (1530), the wearable device (1500) may obtain spatial information (e.g., a depth map) for at least a portion of the FoV of a user wearing the wearable device (1500). Although not shown, a microphone may be disposed on the second face (1520) of the wearable device (1500) to obtain a sound output from an external object. The number of microphones may be one or more, depending on the embodiment.

[0230] In one embodiment, a method may be required to notify the user of the movement of the wearable device approaching the boundary of a safety zone set to provide a virtual space in response to the movement of the wearable device approaching the boundary. As described above, according to one embodiment, a wearable device (e.g., wearable device (101) of FIG. 1 ) may include displays (e.g., display (220) of FIG. 2 ) configured to be positioned toward the eyes of a user wearing the wearable device (e.g., user (110) of FIG. 1 ), at least one camera (e.g., camera (225) of FIG. 2 ) configured to be positioned toward the external environment), at least one sensor (e.g., sensor (230) of FIG. 2 ) configured to obtain information about the external environment, a memory (e.g., memory (215) of FIG. 2 ) including one or more storage media for storing instructions, and at least one processor (e.g., processor (210) of FIG. 2 ) including a processing circuit. The instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to set a safety zone (e.g., safety zone (130) of FIG. 1, safety zone (910) of FIGS. 9A to 9C) with respect to a space of the external environment for the safety of the user wearing the wearable device. The instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to provide a virtual space (e.g., virtual space (920) of FIGS. 9A to 9C) using the displays.The instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to provide, using the displays, a video of the external environment being acquired using the at least one camera and the virtual space together to visually emphasize the external environment with respect to the virtual space based on identifying the wearable device moving out of the safety zone while the virtual space is provided at a first location of the wearable device within the safety zone. The instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to stop providing the virtual space based on identifying the wearable device moving out of the safety zone while the virtual space is provided at a second location of the wearable device within the safety zone. In one embodiment, a wearable device may provide a virtual space within a safety zone for the user's safety, and when the wearable device moves outside the safety zone, the wearable device may continue to provide the virtual space based on whether the virtual space is defined beyond the safety zone. The wearable device may continue to provide a user experience related to the virtual space based on the virtual space defined outside the safety zone after the wearable device moves outside the safety zone. The wearable device may maintain the user experience related to the virtual space based on the state of the external environment outside the safety zone.

[0231] For example, the instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to display a virtual object to determine whether to stop providing the virtual space based on identifying the wearable device moving outside the safety zone while providing the virtual space defined beyond the safety zone.

[0232] For example, the instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to detect a geographic location of the wearable device using the at least one sensor and / or the at least one camera. The instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to compare the safety zone and the geographic location to determine whether the wearable device has moved beyond the safety zone.

[0233] For example, the instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to provide the video being acquired using the at least one camera using the displays based on identifying the wearable device moving outside the safety zone while providing the virtual space defined within the safety zone.

[0234] For example, the instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to provide, using the displays, a virtual plane representing a portion to which the wearable device is approaching, based on identifying the wearable device as being moved within the safety zone toward a portion of the boundary of the safety zone.

[0235] For example, the instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to provide, using the displays, the virtual plane having a color determined using information indicating whether the user can move outside the safety zone through the portion.

[0236] For example, the instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to, while providing the virtual space defined beyond the safety zone, provide a virtual object using the displays to change a mapping between the safety zone and the virtual space based on identifying the wearable device moving toward a boundary of the safety zone adjacent to an external object. The instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to, in response to input related to the virtual object, change the mapping such that a portion of the virtual space defined beyond the safety zone moves into the safety zone.

[0237] For example, the instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to, after changing the mapping in response to the input, display, using the displays, a virtual object indicating a direction of the portion of the virtual space that was defined beyond the boundary prior to changing the mapping.

[0238] For example, the instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to increase the transparency of the virtual space to visually emphasize the external environment relative to the virtual space. For example, the instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to increase the size of an area within the display on which the video is displayed to visually emphasize the external environment relative to the virtual space.

[0239] For example, the instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to increase the size of the region within the display in which the video is displayed, according to a speed indicated by an attribute of at least a portion of the safety zone adjacent the wearable device, while providing the region to visually emphasize the external environment relative to the virtual space.

[0240] In one embodiment, as described above, a non-transitory computer-readable storage medium storing instructions may be provided. The instructions, when executed by a wearable device including displays configured to be positioned toward a user's eyes, at least one camera configured to be positioned toward an external environment, and at least one sensor configured to obtain information about the external environment, may cause the wearable device to set a safety zone in a space of the external environment for the safety of a user wearing the wearable device. The instructions, when executed by the wearable device, may cause the wearable device to provide a virtual space using the displays. The instructions, when executed by the wearable device, may cause the wearable device to provide, using the displays, a video of the external environment being acquired using the at least one camera and the virtual space together, by increasing the transparency of the virtual space, based on identifying the wearable device moving out of the safety zone while the virtual space is provided at a first location of the wearable device within the safety zone. The instructions, when executed by the wearable device, may cause the wearable device to stop providing the virtual space, based on identifying the wearable device moving out of the safety zone while the virtual space is provided at a second location of the wearable device within the safety zone.

[0241] For example, the instructions, when executed by the wearable device, may cause the wearable device to display a virtual object to determine whether to stop providing the virtual space based on identifying the wearable device moving outside the safety zone while providing the virtual space defined beyond the safety zone.

[0242] For example, the instructions, when executed by the wearable device, may cause the wearable device to detect a geographic location of the wearable device using the at least one sensor and / or the at least one camera. The instructions, when executed by the wearable device, may cause the wearable device to compare the safety zone and the geographic location to determine whether the wearable device has moved beyond the safety zone.

[0243] For example, the instructions, when executed by the wearable device, may cause the wearable device to provide, using the displays, the video being acquired using the at least one camera based on identifying the wearable device moving outside the safety zone while providing the virtual space defined within the safety zone.

[0244] For example, the instructions, when executed by the wearable device, may cause the wearable device to use the displays to provide a virtual plane representing a portion to which the wearable device is approaching based on identifying the wearable device moving toward a portion of the boundary of the safety zone within the safety zone.

[0245] For example, the instructions, when executed by the wearable device, may cause the wearable device to provide, using the displays, the virtual plane having a color determined using information indicating whether the user can move outside the safety zone through the portion.

[0246] For example, the instructions, when executed by the wearable device, may cause the wearable device to provide a virtual object using the displays to change a mapping between the safety zone and the virtual space based on identifying the wearable device moving toward a boundary of the safety zone adjacent to an external object while providing the virtual space defined beyond the safety zone. The instructions, when executed by the wearable device, may cause the wearable device, in response to an input related to the virtual object, to change the mapping such that a portion of the virtual space defined beyond the safety zone moves into the safety zone.

[0247] For example, the instructions, when executed by the wearable device, may cause the wearable device to, after changing the mapping in response to the input, display, using the displays, a virtual object indicating a direction of the portion of the virtual space that was defined beyond the boundary before changing the mapping.

[0248] As described above, in one embodiment, a method of a wearable device may be provided. The wearable device may include displays configured to be positioned toward the eyes of a user wearing the wearable device, at least one camera configured to be positioned toward an external environment, and at least one sensor configured to acquire information about the external environment. The method may include an operation of providing a first portion of the virtual space using the displays while the wearable device is positioned within a safety zone set for a space of the external environment for the safety of a user wearing the wearable device. The method may include an operation of ceasing to provide the virtual space using the displays and providing a video of the external environment, which is being acquired using the at least one camera, using the displays based on the wearable device being moved out of the safety zone through a first boundary, which is a portion of a boundary of the safety zone. The method may include an operation of identifying a second portion of the virtual space based on a location of the wearable device outside the safety zone, based on the wearable device moving out of the safety zone through a second boundary that is part of the boundary of the safety zone and different from the first boundary. The method may include an operation of providing a video of the external environment and the second portion of the virtual space using the displays, based on the identification of the second portion of the virtual space.

[0249] For example, the operation of providing the first portion of the virtual space may include an operation of detecting a geographic location of the wearable device using the at least one sensor and / or the at least one camera. The operation of providing the first portion of the virtual space may include an operation of comparing the safety zone and the geographic location to determine whether the wearable device is moved outside the safety zone.

[0250] For example, the operation of providing the first portion of the virtual space may include an operation of providing a first virtual object of a first color representing the first boundary using the displays based on detecting the wearable device moving toward the first boundary. The operation of providing the first portion of the virtual space may include an operation of providing a second virtual object of a second color representing the second boundary using the displays based on detecting the wearable device moving toward the second boundary.

[0251] For example, the action of providing the second part of the virtual space may include an action of providing a virtual object using the displays to determine whether to provide the second part using the displays.

[0252] For example, the act of providing the video and the second portion may include an act of providing a blend of the video and the second portion using the displays by applying a specified transparency to at least one of the video or the second portion.

[0253] According to one embodiment, a wearable device as described above may include displays configured to be positioned toward the eyes of a user wearing the wearable device, at least one camera configured to be positioned toward an external environment, at least one sensor configured to obtain information about the external environment, a memory including one or more storage media for storing instructions, and at least one processor including a processing circuit. The instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to provide a first portion of the virtual space using the displays while the wearable device is positioned within a safety zone set for a space of the external environment for the safety of a user wearing the wearable device. The instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to stop providing the virtual space using the displays and to provide, using the displays, a video of the external environment being acquired using the at least one camera, based on the wearable device moving out of the safety zone through a first boundary that is part of the boundaries of the safety zone. The instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to identify, based on a location of the wearable device outside the safety zone, a second portion of the virtual space, based on the wearable device moving out of the safety zone through a second boundary that is part of the boundaries of the safety zone and is different from the first boundary.The instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to provide, using the displays, a video of the external environment and the second portion of the virtual space based on identifying the second portion of the virtual space.

[0254] For example, the instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to detect a geographic location of the wearable device using the at least one sensor and / or the at least one camera. The instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to compare the safety zone and the geographic location to determine whether the wearable device has moved outside the safety zone.

[0255] For example, the instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to provide, using the displays, a first virtual object of a first color representing the first boundary based on detecting the wearable device moving toward the first boundary. The instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to provide, using the displays, a second virtual object of a second color representing the second boundary based on detecting the wearable device moving toward the second boundary.

[0256] For example, the instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to provide a virtual object using the displays to determine whether to provide the second portion using the displays.

[0257] For example, the instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to provide a blend of the video and the second portion using the displays by applying a specified transparency to at least one of the video or the second portion.

[0258] In one embodiment, as described above, a non-transitory computer-readable storage medium storing instructions may be provided. The instructions may be executed by a wearable device, the wearable device including displays configured to be positioned toward a user's eyes, at least one camera configured to be positioned toward an external environment, and at least one sensor configured to obtain information about the external environment. The instructions, when executed by the wearable device, may cause the wearable device to provide a virtual space using the displays while the wearable device is positioned within a safety zone set for the safety of a user wearing the wearable device. The instructions, when executed by the wearable device, may cause the wearable device to display a virtual object for changing at least one of a placement position or a placement direction of the virtual space relative to the safety zone based on identifying, using the at least one camera and the at least one sensor, that the wearable device has moved toward the boundary of the safety zone. The instructions, when executed by the wearable device, may cause the wearable device to, in response to an input related to the virtual object, change at least one of the placement position or the placement direction of the virtual space, and identify a portion of the virtual space based on a position and orientation of the wearable device in the virtual space resulting from the change. The instructions, when executed by the wearable device, may cause the wearable device to provide, using the displays, the identified portion of the virtual space.

[0259] According to one embodiment, a wearable device (e.g., wearable device (101) of FIG. 1 ) as described above may include displays configured to be positioned toward the eyes of a user wearing the wearable device (e.g., user (110) of FIG. 1 ), at least one camera configured to be positioned toward an external environment (e.g., camera (225) of FIG. 2 ), at least one sensor configured to obtain information about the external environment (e.g., sensor (230) of FIG. 2 ), a memory including one or more storage media for storing instructions (e.g., memory (215) of FIG. 2 ), and at least one processor including a processing circuit (e.g., processor (210) of FIG. 2 ). The instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to provide a virtual space (e.g., a virtual space (920) of FIGS. 9A to 9C) using the displays while the wearable device is positioned within a safety zone set for the safety of a user wearing the wearable device (e.g., a safety zone (130) of FIG. 1, a safety zone (910) of FIGS. 9A to 9C). The instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to display a virtual object (e.g., the virtual object (150) of FIG. 1) for changing at least one of a placement position or a placement direction of the virtual space relative to the safety zone based on identifying, using the at least one camera and the at least one sensor, that the wearable device has moved to the boundary of the safety zone.The instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device, in response to an input related to the virtual object, to change at least one of the placement location or the placement direction in the virtual space, and to identify a portion of the virtual space based on a position and orientation of the wearable device in the virtual space resulting from the change. The instructions, when individually and / or collectively executed by the at least one processor, may cause the wearable device to provide the identified portion of the virtual space using the displays.

[0260] As used herein, the term "if" will be understood to mean "when, upon," "in response to determining," or "in response to detecting," depending on the context. Similarly, "if it is determined to," or "if [the stated condition or event] is detected," will optionally be understood to mean "upon determining," or "in response to determining," "upon detecting [the stated condition or event]," or "in response to detecting [the stated condition or event]."

[0261] The devices described above may be implemented as hardware components, software components, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments may be implemented using one or more general-purpose computers or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding to them. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device is sometimes described as being used alone; however, one of ordinary skill in the art will recognize that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, a processing unit may include multiple processors, or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.

[0262] Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing device to perform a desired operation or may independently or collectively command the processing device. The software and / or data may be embodied in any type of machine, component, physical device, computer storage medium, or device for interpretation by the processing device or for providing instructions or data to the processing device. The software may be distributed over networked computer systems and stored or executed in a distributed manner. The software and data may be stored on one or more computer-readable recording media.

[0263] The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. In this case, the medium may be one that continuously stores a computer-executable program or one that temporarily stores it for execution or download. In addition, the medium may be various recording means or storage means in the form of a single or multiple hardware combinations, and is not limited to a medium directly connected to a computer system, but may also be distributed over a network. Examples of the medium may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and those configured to store program commands, including ROM, RAM, and flash memory. In addition, examples of other media may include recording media or storage media managed by app stores that distribute applications, sites that supply or distribute various software, servers, etc.

[0264] Although the embodiments described above have been described by way of limited examples and drawings, those skilled in the art will appreciate that various modifications and variations can be made based on the above teachings. For example, appropriate results can still be achieved even if the described techniques are performed in a different order than described, and / or components of the described systems, structures, devices, circuits, etc. are combined or combined in a different manner than described, or are replaced or substituted with other components or equivalents.

[0265] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims described below.

Claims

1. In wearable devices, Displays configured to be positioned toward the eyes of a user wearing the wearable device; At least one camera configured to be positioned facing the external environment; At least one sensor configured to obtain information about the external environment; A memory comprising one or more storage media for storing instructions; and At least one processor comprising a processing circuit, The above instructions, when individually and / or collectively executed by the at least one processor, cause the wearable device to: To ensure the safety of the user wearing the wearable device, a safety zone is set for the space of the external environment; Providing a virtual space using the above displays; At a first location of the wearable device within the safety zone, while the virtual space is provided beyond the safety zone, based on identifying the wearable device moving out of the safety zone, providing a video of the external environment acquired using the at least one camera and the virtual space together using the displays to visually emphasize the external environment with respect to the virtual space; and While the virtual space is provided at a second location of the wearable device within the safety zone, causing the provision of the virtual space to be stopped based on identifying the wearable device moving out of the safety zone. Wearable devices.

2. In claim 1, the instructions, when individually and / or collectively executed by the at least one processor, cause the wearable device to: While providing the virtual space defined beyond the safety zone, based on identifying the wearable device moving out of the safety zone, causing a virtual object to be displayed to determine whether to stop providing the virtual space. Wearable devices.

3. In claims 1 and 2, when the instructions are individually and / or collectively executed by the at least one processor, the wearable device, Detecting the geographic location of the wearable device using at least one sensor and / or at least one camera; Comparing the above safety zone and the above geographic location to determine whether the wearable device is moved beyond the safety zone, Wearable devices.

4. In claims 1 to 3, the instructions, when individually and / or collectively executed by the at least one processor, cause the wearable device to: While providing the virtual space defined within the safety zone, based on identifying the wearable device moving out of the safety zone, causing the video being acquired using the at least one camera to be provided using the displays. Wearable devices.

5. In claims 1 to 4, the instructions, when individually and / or collectively executed by the at least one processor, cause the wearable device to: Based on identifying the wearable device moving toward a portion of the boundary of the safety zone within the safety zone, a virtual plane representing the portion to which the wearable device approaches is provided using the displays. Wearable devices.

6. In claim 5, the instructions, when individually and / or collectively executed by the at least one processor, cause the wearable device to: Causing the displays to provide a virtual plane having a color determined using information indicating whether the user can move out of the safety zone through the above part. Wearable devices.

7. In claims 1 to 6, the instructions, when individually and / or collectively executed by the at least one processor, cause the wearable device to: While providing the virtual space defined beyond the safety zone, providing a virtual object for changing the mapping between the safety zone and the virtual space using the displays based on identifying the wearable device moving toward the boundary of the safety zone adjacent to an external object; and In response to an input related to said virtual object, causing said mapping to be changed so that a portion of said virtual space defined beyond said safety zone moves into said safety zone. Wearable devices.

8. In claim 7, the instructions, when individually and / or collectively executed by the at least one processor, cause the wearable device to: causing said displays to display a virtual object indicating the direction of said portion of said virtual space that was defined beyond said boundary before changing said mapping in response to said input; Wearable devices.

9. In claim 1, the instructions, when individually and / or collectively executed by the at least one processor, cause the wearable device to: To visually emphasize the external environment in relation to the above virtual space: Increase the transparency of the virtual space; or causing the size of the area within the display in which the above video is displayed to increase, Wearable devices.

10. In claim 9, the instructions, when individually and / or collectively executed by the at least one processor, cause the wearable device to: In order to visually emphasize the external environment with respect to the virtual space, while providing an area within the display where the video is displayed, causing the size of said area to increase according to a speed indicated by an attribute of at least a portion of said safety zone adjacent to said wearable device; Wearable devices.

11. A non-transitory computer-readable storage medium storing instructions, wherein the instructions, when executed by a wearable device, include displays configured to be positioned toward a user's eyes, at least one camera configured to be positioned toward an external environment, and at least one sensor configured to obtain information about the external environment, wherein the wearable device: To ensure the safety of the user wearing the wearable device, a safety zone is set for the space of the external environment; Providing a virtual space using the above displays; At a first location of the wearable device within the safety zone, while the virtual space is provided beyond the safety zone, based on identifying the wearable device moving out of the safety zone, by increasing the transparency of the virtual space, a video of the external environment acquired using the at least one camera and the virtual space are provided together using the displays; While the virtual space is provided at a second location of the wearable device within the safety zone, causing the provision of the virtual space to be stopped based on identifying the wearable device moving out of the safety zone. Non-transitory computer-readable storage medium.

12. In claim 11, the instructions, when executed by the wearable device, cause the wearable device to: While providing the virtual space defined beyond the safety zone, based on identifying the wearable device moving out of the safety zone, causing a virtual object to be displayed to determine whether to stop providing the virtual space. Non-transitory computer-readable storage medium.

13. In claims 11 to 12, the instructions, when executed by the wearable device, cause the wearable device to: Detecting the geographic location of the wearable device using at least one sensor and / or at least one camera; Comparing the above safety zone and the above geographic location to determine whether the wearable device is moved beyond the safety zone, Non-transitory computer-readable storage medium.

14. In claims 11 to 13, the instructions, when executed by the wearable device, cause the wearable device to: While providing the virtual space defined within the safety zone, based on identifying the wearable device moving out of the safety zone, causing the video being acquired using the at least one camera to be provided using the displays. Non-transitory computer-readable storage medium.

15. In claims 11 to 14, the instructions, when executed by the wearable device, cause the wearable device to: Based on identifying the wearable device moving toward a portion of the boundary of the safety zone within the safety zone, a virtual plane representing the portion to which the wearable device approaches is provided using the displays. Non-transitory computer-readable storage medium.

Images