Wearable device, method, and non-transitory computer-readable storage medium for authorizing user to view external object

Abstract

This wearable device comprises: a memory storing instructions and including one or more storage media; one or more cameras; a display; and at least one processor including processing circuitry, wherein the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to: display an immersive environment on the display; detect, through the one or more cameras, an external object in front of the wearable device worn by a user; on the basis of the external object detected while displaying the immersive environment, set a safety zone located within a threshold distance from the external object; identify a state of the safety zone; and on the basis of a change in the state of the safety zone identified while displaying the immersive environment, display, on the display, a visual object corresponding to the external object and obtained through the one or more cameras so as to authorize the user to view the external object through a portion of the immersive environment.

Description

Wearable device, method, and non-transitory computer-readable storage medium for authorizing a user to view an external object

[0001] The present disclosure relates to a wearable device, a method, and a non-transitory computer-readable storage medium for authorizing a user to view an external object.

[0002] To provide an enhanced user experience, electronic devices are being developed that provide augmented reality (AR) services, which display computer-generated information in conjunction with external objects in the real world. These electronic devices may be wearable devices worn by the user. For example, these electronic devices may be AR glasses and / or head-mounted devices (HMDs).

[0003] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art related to the present disclosure.

[0004] A wearable device is described. The wearable device may include at least one processor comprising a memory storing instructions and including one or more storage media, one or more cameras, a display, and a processing circuit. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display an immersive environment on the display. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to detect, via the one or more cameras, an external object in front of the wearable device worn by a user. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to establish a safety zone located within a threshold distance from the external object based on the external object detected while displaying the immersive environment. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify a state of the safety zone. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display, on the display, a visual object corresponding to the external object and acquired through the one or more cameras to authorize the user to view the external object through a portion of the immersive environment, based on a change in the state of the identified safety zone while displaying the immersive environment.

[0005] A method is described. The method may be performed in a wearable device comprising one or more cameras and a display. The method may include displaying an avatar representing a user on the display assembly. The method may include displaying an immersive environment on the display. The method may include detecting an external object in front of the wearable device worn by the user through the one or more cameras. The method may include establishing a safety zone located within a threshold distance from the external object based on the external object detected while displaying the immersive environment. The method may include identifying a state of the safety zone. The method may include displaying a visual object corresponding to the external object and acquired through the one or more cameras on the display based on a change in the state of the safety zone identified while displaying the immersive environment to authorize the user to view the external object through a portion of the immersive environment.

[0006] A non-transitory computer-readable storage medium is described. The non-transitory computer-readable storage medium may store one or more programs. The one or more programs may include instructions that, when executed by a wearable device including one or more cameras and a display, cause the wearable device to display an immersive environment on the display. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to detect, through the one or more cameras, an external object in front of the wearable device worn by a user. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to set a safety zone located within a threshold distance from the external object based on the external object detected while displaying the immersive environment. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to identify a state of the safety zone. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to display, on the display, a visual object corresponding to the external object and acquired through the one or more cameras to authorize the user to view the external object through a portion of the immersive environment, based on a change in the state of the safety zone identified while displaying the immersive environment.

[0007] Figure 1 illustrates an example of a visual object corresponding to an external object that is not displayed within an immersive environment.

[0008] Figure 2 is a simplified block diagram of an exemplary wearable device.

[0009] FIG. 3 is a flowchart illustrating exemplary operations of a wearable device for displaying a visual object based on a change in the state of a safety zone.

[0010] Figure 4 illustrates an example of a safety zone for external objects.

[0011] Figures 5a, 5b, 5c, and 5d illustrate examples of changes in the state of the safety zone.

[0012] Figure 6 illustrates an example of displaying a visual object based on a change in the status of a safety zone.

[0013] Figure 7 illustrates an example of displaying a rendering object.

[0014] Figure 8 illustrates an example of a safety zone for an external object with repetitive movement.

[0015] Figure 9 illustrates examples of components of a wearable device.

[0016] FIG. 10 is a block diagram of an electronic device within a network environment according to various embodiments.

[0017] Figures 11a and 11b show an example of the appearance of a wearable device.

[0018] Figure 12 shows an example of a block diagram of a wearable device.

[0019] Fig. 13 shows an example of a block diagram of an electronic device for displaying an image in a virtual space.

[0020] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0021] Figure 1 illustrates an example of a visual object corresponding to an external object that is not displayed within an immersive environment.

[0022] Referring to FIG. 1, a wearable device (100) may include a head-mounted display (HMD) that can be worn on the head of a user (105). The wearable device (100) may be described as a head-mounted display (HMD) device, a headgear electronic device, a glasses-type (or goggle-type) electronic device, a video see-through (VST) device, an extended reality (XR) device, a virtual reality (VR) device, and / or an augmented reality (AR) device.

[0023] The wearable device (100) may include a display (e.g., the display (240) of FIG. 2). For example, the display may be configured to be positioned toward the eyes of a user (105) wearing the wearable device (100). For example, the wearable device (100) may display or provide an immersive environment (120) through the display. For example, the immersive environment (120) may include a virtual reality environment, a virtual space, and / or a simulated space. For example, the immersive environment (120) may be described as an environment for providing functions related to augmented reality (AR) and / or mixed reality (MR). For example, a wearable device (100) can provide a user experience that is separated (or disconnected) from the external environment by displaying an immersive environment that is different from the external environment.

[0024] External objects (110, 115) may be located within the external environment (or physical environment, or physical world) in which the user (105) is located. For example, the external object (110) may be movable or may collide with (or come into contact with) another external object (115) (or the user (105)). For example, the wearable device (100) may refrain from displaying (or not displaying) a visual object (130) corresponding to (or representing) the external object (110) and / or another visual object (125) corresponding to (or representing) another external object (115) within the immersive environment (120). For example, while an immersive environment (120) is displayed, a user (105) may not perceive movement of an external object (110) and collisions between the external object (110) and another external object (115) as visual objects (130) and / or other visual objects (125) are not displayed.

[0025] For example, a collision between an external object (110) and another external object (115) may cause a risk to the user (105). For example, the user (105) may feel a risk or discomfort due to not recognizing the movement of the external object (110) and / or the collision between the external object (110) and another external object (115). A method may be required to resolve the risk and discomfort caused by the user (105) not recognizing the movement of the external object (110) and / or the collision between the external object (110) and another external object (115). To alleviate the risk and discomfort of such users (105), the wearable device (100) may display a visual object (130) based on an external object (110) moving within the external environment, or may display a visual object (130) and another visual object (125) based on a collision between an external object (110) and another external object (115) caused within the external environment.

[0026] The wearable device (100) may perform the operations exemplified in the descriptions of FIGS. 3 to 8 to display a visual object (130) and / or another visual object (125). The wearable device (100) may include components for performing the operations. The components may be exemplified in the description of FIG. 2.

[0027] Figure 2 is a simplified block diagram of an exemplary wearable device.

[0028] Referring to FIG. 2, the wearable device (200) may be described as a head-mounted display (HMD) device, a headgear electronic device, a glasses-type (or goggle-type) electronic device, a video see-through (VST) device, an extended reality (XR) device, a virtual reality (VR) device, and / or an augmented reality (AR) device that can be worn on a user's head. An example of the structure of the wearable device (200) that can be worn on a user's head is described with reference to FIG. 11A and / or FIG. 11B. The wearable device (200) may include at least a part of the electronic device (1001) of FIG. 10, or may correspond to at least a part of the electronic device (1001) of FIG. 10. A wearable device (200) may include at least one processor (210), memory (220), one or more cameras (230), and a display (240).

[0029] At least one processor (210) may include processing circuitry. At least one processor (210) may include a central processing unit (CPU) (e.g., including processing circuitry). At least one processor (210) may include a graphic processing unit (GPU) (e.g., including processing circuitry) and a neural processing unit (NPU) (e.g., including processing circuitry). For example, at least one processor (210) may be configured to control a memory (220), one or more cameras (230), and a display (240). At least one processor (210) may be configured to individually or collectively execute instructions stored in the memory (220) to cause the wearable device (200) (or the wearable device (100)) to perform at least some of the operations illustrated in the description of FIG. 1. At least one processor (210) may be configured to individually or collectively execute instructions stored in the memory (220) to cause the wearable device (200) to perform at least some of the operations exemplified in the descriptions of FIGS. 3 through 8.

[0030] The memory (220) may include one or more storage media. The memory (220) may store various data used by at least one component of the wearable device (200) (e.g., at least one processor (210), the memory (220), one or more cameras (230), and / or the display (240)). For example, the data may include input data or output data for software and commands related thereto. The memory (220) may include volatile memory or non-volatile memory.

[0031] The one or more cameras (230) may include one or more optical sensors (e.g., a charged coupled device (CCD) sensor, a complementary metal oxide semiconductor (CMOS) sensor) that generate an electrical signal representing the color and / or brightness of light. For example, the one or more cameras (230) may be described as an image sensor. For example, the one or more cameras (230) may be available to acquire images of the space (or external environment) in front of the wearable device (200). For example, at least some of the one or more cameras (230) may have a field of view (FOV) corresponding to the FOV of the user's eye. For example, the FOV of some of the one or more cameras (230) may be different from the FOV of other some of the one or more cameras (230).

[0032] The display (240) may be configured to visualize information (or signals) provided from at least one processor (210). The display (240) may be positioned to face the eyes of a user wearing the wearable device (200). The display (240) may be configured to display an immersive environment and visual objects. The display (240) may include at least one display.

[0033] The wearable device (200) illustrated in the description of FIG. 2 can execute at least some of the operations illustrated in the description of FIGS. 3 to 8. The operations illustrated in the description of FIGS. 3 to 8 can be caused by (or within) the wearable device (200) under the control of at least one processor (210).

[0034] FIG. 3 is a flowchart illustrating exemplary operations of a wearable device for displaying a visual object based on a change in the state of a safety zone.

[0035] Referring to FIG. 3, in operation 300, at least one processor (210) may display (or provide) an immersive environment (or virtual reality environment, or virtual space, or simulated environment) (e.g., the immersive environment (120) of FIG. 1) on a display (240). For example, the immersive environment may be described as an environment for providing functions related to augmented reality and / or mixed reality. For example, the at least one processor (210) may provide a user experience that is separated (or disconnected) from the external environment by displaying an immersive environment that is different from the external environment. For example, the at least one processor (210) may display stereoscopic scenes to create an illusion of depth within the immersive environment.

[0036] In operation 310, at least one processor (210) may detect an external object (e.g., an external object (410) of FIG. 4) in front of a wearable device (200) worn by a user through one or more cameras (230). For example, at least one processor (210) may detect an external object located within a field of view (FOV) of one or more cameras (230). For example, at least one processor (210) may detect an external object using at least a portion of images acquired through one or more cameras (230). For example, at least one processor (210) may recognize or track one or more external objects in an external environment through one or more cameras (230).

[0037] In operation 320, at least one processor (210) may set a safety zone located within a threshold distance from an external object based on an external object detected while displaying an immersive environment. The safety zone for an external object is exemplified in the description of FIG. 4.

[0038] Figure 4 illustrates an example of a safety zone for external objects.

[0039] Referring to FIG. 4, an external object (410) may be located within an external environment in front of a wearable device (200) worn by a user (400). For example, at least one processor (210) may detect the external object (410) through one or more cameras (230). For example, at least one processor (210) may set a safety zone (405) for the external object (410) based on the detected external object (410) while displaying an immersive environment. For example, the safety zone (405) may be located within a threshold distance from the external object (410). For example, the safety zone (405) may surround the external object (410). For example, the safety zone (405) may be set to identify movement of the external object (410) or collisions between external objects. For example, at least one processor (210) may set a safety zone (405) based on characteristics of an external object (410) (e.g., size, posture, movement, distance to the wearable device (200), and / or action). For example, the safety zone (405) may not be displayed within the immersive environment, but rather, the external environment within the safety zone (405) may be shown to the user (400) based on satisfying certain conditions. For example, the certain conditions may include a change in the state of the safety zone (405).

[0040] The safety zone (405) may be established or defined to transition between the immersive environment and the external environment. For example, the safety zone (405) may be referred to as a protection area, a protection area, a geographical area, a safety area, a safety area, a guardian area, a guardian area, and / or a guardian. For example, the boundary (or virtual boundary) of the safety zone (405) may be referred to as a fence and / or a wall in terms of the boundary separating it from the external environment. For example, although the safety zone (405) is illustrated as having a hexahedral shape in FIG. 4, the safety zone (405) may have various shapes. For example, the shape of the safety zone (405) may correspond to the shape of the external object (410) or may be determined based on the shape of the external object (410).

[0041] In another embodiment, at least one processor (210) may identify the type of the external object (410) based on detecting the external object (410) via one or more cameras (230). For example, the at least one processor (210) may determine whether to set a safety zone (405) for the external object (410) based on the identified type of the external object (410). For example, the at least one processor (210) may set a safety zone (405) for the external object (410) based on a type of the external object (410) that corresponds to a reference type. For example, the at least one processor (210) may refrain from (or stop, or bypass, or not set) setting a safety zone (405) for the external object (410) based on a type of the external object (410) that is different from the reference type. For example, the criteria type may include types of external objects that are dangerous to the user, fragile, breakable, or collide with the user. For example, the criteria type may be predetermined by the user (400). For example, at least one processor (210) may change, add, or remove the criteria type for determining whether to set a safety zone (405) based on the input of the user (400) or the gesture of the user (400). For example, at least one processor (210) may refrain from setting a safety zone for external objects that are not dangerous to the user (400) by excluding the types of non-hazardous external objects from the criteria type based on the input (or user gesture) of the user (400).

[0042] As a non-limiting example, at least one processor (210) may set a safety zone for an external object based on a user's (400) input (or user gesture) regarding the external object, even if the external object is not detected by one or more cameras (230). However, this is not limiting.

[0043] Referring again to FIG. 3 , at operation 330, at least one processor (210) may identify a state of a safety zone. For example, at least one processor (210) may identify whether the state of the safety zone changes while displaying an immersive environment. Changes in the state of the safety zone are exemplified in the descriptions of FIGS. 5A , 5B , 5C , and 5D .

[0044] Figures 5a, 5b, 5c, and 5d illustrate examples of changes in the state of the safety zone.

[0045] Referring to FIG. 5A, an external object (500) in which a safety zone (510) is set may move within an external environment. For example, at least one processor (210) may identify movement of an external object (500) in front of a wearable device (200) through one or more cameras (230). For example, at least one processor (210) may track the external object (500) using one or more cameras (230), thereby identifying movement of the external object (500) based on a change in a motion vector of the external object (500). For example, at least one processor (210) may move the safety zone (510) for the external object (500) as the external object (500) moves. For example, as the safety zone (510) moves, at least a portion of an image within the safety zone (510) may change. For example, at least one processor (210) can identify a change in the state of the safety zone (510) based on at least a portion of an image within the safety zone (510) that has changed as the safety zone (510) moves.

[0046] At least one processor (210) can identify a direction (505) in which an external object (500) moves through one or more cameras (230). For example, at least one processor (210) can expand a safety zone (510) in the direction (505) based on the direction (505) in which the external object (500) moves. For example, by expanding the safety zone (510) in the direction (505) in which the external object (500) moves, the user (400) can relatively quickly recognize a change in the safety zone in the direction (505) in which the external object (500) moves. Since the external object (500) is relatively quickly recognized by the user (400), the external object (500) can be shown to the user (400) before colliding with another external object (or the user (400)). For example, the safety zone (510) may change its state by expanding in the direction (505). For example, at least one processor (210) may identify a change in the state of the safety zone (510) due to expansion of the safety zone (510) in the direction (505).

[0047] At least one processor (210) can identify the speed (or motion vector) of an external object (500) through one or more cameras (230). For example, at least one processor (210) can determine the extent to which the safety zone (510) is expanded based on the speed (or motion vector) of the external object (500). For example, at least one processor (210) can expand the safety zone (510) in proportion to the speed (or motion vector) of the external object (500) in the direction of movement (505) of the external object (500). For example, for an external object (500) having a relatively large speed, it may be required to notify the user (400) in advance of a collision between the external object (500) and another external object (or the user (400)). For example, by expanding the safety zone (510) in proportion to the speed (or motion vector) of the external object (500), the external object (500) can be notified to the user (400) relatively quickly before the external object (500) collides with another external object (or the user (400)).

[0048] Referring to FIG. 5B, within the external environment, another external object (525) may move toward the external object (515). For example, the other external object (525) may be described as an external object for which a safety zone is not set, and the external object (515) may be described as an external object for which a safety zone (520) is set. For example, at least one processor (210) may detect another external object (525) located within the safety zone (520) by moving toward the external object (515).

[0049] As a non-limiting example, where an external object (515) (e.g., a trash can) includes a portion (e.g., an entrance of the trash can) that can interact with another external object (525) (e.g., trash), at least one processor (210) can set a safety zone (520) having a relatively wide (or large) portion thereof.

[0050] For example, at least a portion of an image within the safety zone (520) (e.g., a portion within the safety zone (520) where the other external object (525) is positioned) may be changed as another external object (525) is positioned within the safety zone (520). For example, at least one processor (210) may identify a change in the state of the safety zone (520) based on at least a portion of an image within the safety zone (520) that is changed as the other external object (525) moves within the safety zone (520).

[0051] Referring to FIG. 5C, within an external environment, a body part (540) of a user (400) may move toward an external object (530). For example, the user (400) may move a body part (540) of the user (400) toward an external object (530) to grip the external object (530) or to use the external object (530). For example, at least one processor (210) may detect a body part (540) of the user (400) located within a safety zone (535) by moving the body part (540) of the user (400) toward the external object (530).

[0052] For example, at least a portion of an image within the safety zone (535) may be changed as a part of the body (540) of the user (400) is positioned within the safety zone (535). For example, at least one processor (210) may identify a change in the state of the safety zone (535) based on at least a portion of an image within the safety zone (535) that is changed as a part of the body (540) of the user (400) moves within the safety zone (535).

[0053] Referring to FIG. 5D, within an external environment, an external object (500) may move toward an external object (515). For example, each of the external object (500) and the external object (515) may be described as an external object in which a safety zone (510) and a safety zone (520) are set. For example, at least one processor (210) may identify the movement of the external object (500) in front of the wearable device (200) through one or more cameras (230). For example, at least one processor (210) may identify the movement of the external object (500) by tracking the external object (500) using one or more cameras (230). For example, at least one processor (210) may move the safety zone (510) for the external object (500) as the external object (500) moves. For example, as the safety zone (510) moves, at least a portion of the image within the safety zone (510) may change. For example, at least one processor (210) may identify a change in the state of the safety zone (510) based on at least a portion of the image within the safety zone (510) that changes as the safety zone (510) moves.

[0054] For example, as the safety zone (510) moves, at least a portion of the safety zone (510) may overlap at least a portion of the safety zone (520). For example, at least one processor (210) may identify that at least a portion of the safety zone (510) overlaps at least a portion of the safety zone (520). For example, the state of the safety zone (510) and the state of the safety zone (520) may change as at least a portion of the safety zone (510) overlaps at least a portion of the safety zone (520). For example, at least one processor (210) may identify a change in the state of the safety zone (510) and the state of the safety zone (520) based on that at least a portion of the safety zone (510) overlaps at least a portion of the safety zone (520).

[0055] Referring again to FIG. 3 , at operation 340, at least one processor (210) may display a visual object on the display (240) based on a change in the state of the identified safety zone while displaying the immersive environment. For example, the visual object may correspond to an external object. For example, the visual object may be acquired through one or more cameras (230). For example, at least one processor (210) may display the visual object to allow the user to view the external object through a portion of the immersive environment. For example, the visual object may be defined as an object corresponding to an external object displayed through video see-through. The visual object displayed based on a change in the state of the safety zone is exemplified in the description of FIG. 6 .

[0056] Figure 6 illustrates an example of displaying a visual object based on a change in the status of a safety zone.

[0057] Referring to FIG. 6, at least one processor (210) may display (or provide) an immersive environment (600) via a display (240). For example, while displaying the immersive environment (600), at least one processor (210) may identify a state of the safety zone (510) and a change in the state of the safety zone (520) due to at least a portion of the safety zone (510) overlapping at least a portion of the safety zone (520). FIG. 6 illustrates a state of the safety zone (510) and a change in the state of the safety zone (520) as at least a portion of the safety zone (510) overlaps at least a portion of the safety zone (520), but may include changes in the state of the safety zones (510, 520, 535) exemplified in the descriptions of FIGS. 5a, 5b, 5c, and 5d.

[0058] At least one processor (210) may display a visual object (605) and a visual object (610) on the display (240) based on a change in the state of the safety zone (510) and a state of the safety zone (520). For example, the visual object (605) may correspond to an external object (500), and the visual object (610) may correspond to an external object (515). For example, the at least one processor (210) may acquire the visual object (605) and the visual object (610) through one or more cameras (230). For example, the visual object (605) and the visual object (610) may be derived from a shape of the external object (500) and a shape of the external object (515) included in at least a portion of images acquired through the one or more cameras (230). For example, at least one processor (210) may display a visual object (605) and a visual object (610) to allow a user (400) to view an external object (500) and an external object (515) through a portion of an immersive environment (600). For example, at least one processor (210) may display, within the immersive environment (600), a first area (615) representing an external environment within a safety zone (510) and a second area (620) representing an external environment within a safety zone (520). For example, the first area (615) and the second area (620) may be acquired via one or more cameras (230). For example, the at least one processor (210) may display the first area (615) and the second area (620) via video see through. For example, at least one processor (210) may authorize a user (400) to view an external object (500) and an external object (515) by displaying a first area (615) and a second area (620) via video see through.

[0059] For example, by displaying a visual object (605) and a visual object (610), at least one processor (210) can be authorized to recognize the movement of an external object (500) and the collision of the external object (500) with the external object (515) while the immersive environment (600) is displayed to the user (400). For example, by allowing the user (400) to recognize in advance that the external object (500) is colliding with the external object (515), a risk resulting from a collision between the external object (500) and the external object (515) can be prevented or resolved.

[0060] For example, at least one processor (210) may display a visual object corresponding to an external object (e.g., an external object (530) of FIG. 5C) based on a change in the state of a safety zone due to a part of the user's (400) body (e.g., a part (540) of FIG. 5C) being located within the safety zone (e.g., a safety zone (535) of FIG. 5C). For example, at least one processor (210) may authorize the user (400) to recognize the external object while the immersive environment (600) is displayed by displaying the visual object. For example, the user (400) may grasp or use the external object by recognizing the external object while the immersive environment (600) is displayed.

[0061] According to another embodiment, at least one processor (210) may display a first area (615) and a second area (620) on the display (240) based on a state of the safety zone (510) and a change in the state of the safety zone (520). For example, the first area (615) may correspond to the safety zone (510), and the second area (620) may correspond to the safety zone (520). For example, the at least one processor (210) may acquire the first area (615) and the second area (620) through one or more cameras (230). For example, the first area (615) and the second area (620) may be derived from images within the safety zone (510) and the safety zone (520) included in at least some of the images acquired through the one or more cameras (230). For example, at least one processor (210) may display a first region (615) and a second region (620) to authorize the user (400) to view the external object (500) and the external object (515) through a portion of the immersive environment (600). For example, by displaying the first region (615) and the second region (620), the at least one processor (210) may authorize the user (400) to view the external object (500) and the external object (515) through a larger portion of the immersive environment (600) than by displaying the visual object (605) and the visual object (610). For example, at least one processor (210) may display images within a safety zone (510) and images within a safety zone (520) acquired through one or more cameras (230) through a display (240), thereby making external objects (500) and external objects (515) visible to the user (400) through a wider portion of a relatively immersive environment (600), thereby preventing or alleviating a risk arising from a collision between the external objects (500) and external objects (515).

[0062] For example, while the visual object (605) and / or the visual object (610) is displayed, the external object (500) and / or the external object (515) may not move within the external environment. For example, at least one processor (210) may identify that the state of the safety zone (510) and / or the state of the safety zone (520) is maintained by the external object (500) and / or the external object (515) not moving. For example, at least one processor (210) may identify whether the state of the safety zone (510) and / or the state of the safety zone (520) is maintained during the reference time.

[0063] At least one processor (210) may stop (or refrain from, or not display) the visual object (605) and / or the visual object (610) based on the state of the safety zone (510) and / or the state of the safety zone (520) maintained for a reference period of time. For example, by not moving the external object (500) and / or the external object (515), the risk to the user (400) from the external object (500) and / or the external object (515) within the external environment may be reduced. For example, by stopping displaying the visual object (605) and / or the visual object (610), the at least one processor (210) may provide the user (400) with an immersive environment (600) that is disconnected from the external environment.

[0064] For example, at least one processor (210) can perform rendering processing on a visual object (605, 610). Based on the rendering processing on the visual object (605, 610), the obtained rendering model is exemplified in the description of FIG. 7.

[0065] Figure 7 illustrates an example of displaying a rendering object.

[0066] Referring to FIG. 7, at least one processor (210) may detect an external object (530) through one or more cameras (230) while displaying an immersive environment (600). For example, based on detecting the external object (530), the at least one processor (210) may perform rendering processing on a visual object corresponding to the external object (530) and acquired through one or more cameras (230). For example, the at least one processor (210) may acquire a rendering object (700) by performing rendering processing on the visual object. For example, the rendering object (700) may be described as a 3D (three dimensional) object having a shape similar to or approximately similar to the shape of the external object (530). For example, as the external object (530) has various shapes, the rendering objects stored in the memory (220) of the wearable device (200) may not include a rendering object corresponding to the shape of the external object (530). For example, since the rendering objects stored in the memory (220) do not include a rendering object corresponding to the shape of the external object (530), at least one processor (210) may determine a rendering object (700) having a shape similar (or approximate) to the shape of the external object (530) among the rendering objects stored in the memory (220).

[0067] For example, at least one processor (210) may display a rendering object (700) at a location within the immersive environment (600) where a visual object is to be displayed. For example, at least one processor (210) may display a rendering object (700) at a location within the immersive environment (600) that corresponds to a location of an external object (530) within the external environment. For example, by displaying the rendering object (700) within the immersive environment (600), the at least one processor (210) may authorize the user (400) to recognize the location of the external object (530) and an approximate shape of the external object (530) while the immersive environment (600) is displayed. For example, by the user (400) recognizing the location of the external object (530) and an approximate shape of the external object (530), the user (400) may grasp or otherwise use the external object (530) while the immersive environment (600) is displayed.

[0068] For example, at least one processor (210) can detect a part of a user's (400) body (540) located within a safety zone (535) by moving the part of the user's (400) body (540) toward an external object (530). For example, at least one processor (210) can identify a change in the state of the safety zone (535) based on at least a portion of an image within the safety zone (535) that changes as the part of the user's (400) body (540) moves into the safety zone (535). For example, at least one processor (210) can stop displaying a rendering object (700) and display a visual object corresponding to the external object (530) based on the change in the state of the safety zone (535).

[0069] For example, an external object may have repetitive (or periodic) movement. A safety zone for an external object with repetitive (or periodic) movement is exemplified in the description of FIG. 8.

[0070] Figure 8 illustrates an example of a safety zone for an external object with repetitive movement.

[0071] Referring to FIG. 8, within a state (800), an external object (805) may move (or rotate) in a direction (815). At least one processor (210) may expand a safety zone (810) for the external object (805) in the direction (815) based on identifying the external object (805) moving (or rotating) in the direction (815) through one or more cameras (230).

[0072] The external object (805) may have a repetitive (or periodic) movement by moving (or rotating) in the direction (815) and then moving (or rotating) in the direction (830). The wearable device (200) may transition from state (800) to state (825) by at least one processor (210) identifying, through one or more cameras (230), that the external object (805) moves (or rotates) in the direction (815) and then moves (or rotates) in the direction (830). Within the state (800), the at least one processor (210) may expand the safety zone (810) for the external object (805) in the direction (830) based on identifying that the external object (805) moves (or rotates) in the direction (830).

[0073] Within the state (800) and the state (825), the state of the safety zone (810) may change as the safety zone (810) expands or moves. At least one processor (210) may identify a change in the state of the safety zone (810) while displaying the immersive environment (600). For example, if at least one processor (210) displays a visual object (820) corresponding to the external object (805) based on a change in the state of the safety zone (810) due to repetitive (or periodic) movement of the external object (805), the visual object (820) may continue to be displayed within the immersive environment (600). Continuously displaying a visual object (820) in response to repetitive (or periodic) movement of an external object (805) may increase power consumption within the wearable device (200) or may not be considered a risk factor for the user (400).

[0074] For example, at least one processor (210) may refrain from displaying (or stop, or not display) a visual object (820) despite a change in the state of the safety zone (810) due to repetitive (or periodic) movement of an external object (805). For example, by refraining from displaying the visual object (820), the at least one processor (210) may reduce power consumption within the wearable device (200). For example, even if the at least one processor (210) refrain from displaying the visual object (820), the user (400) may feel safe or not uncomfortable.

[0075] Figure 9 illustrates examples of components of a wearable device.

[0076] Referring to FIG. 9, the wearable device (200) may include an object recognition unit (900), a virtual object rendering unit (910), a control unit (920), a safety zone allocation unit (930), and a video see-through unit (940). The object recognition unit (900), the virtual object rendering unit (910), the control unit (920), the safety zone allocation unit (930), and the video see-through unit (940) may support a function of processing a visual object through an algorithm stored in the memory (220). Although the object recognition unit (900), the virtual object rendering unit (910), the control unit (920), the safety zone allocation unit (930), and the video see-through unit (940) are described as ‘units’, they may perform the following functions in software and / or functionally.

[0077] The object recognition unit (900) can detect (or recognize) external objects located within the FOV of one or more cameras (230) using one or more cameras (230). For example, the object recognition unit (900) can store information about the detected external objects in the memory (220). For example, the object recognition unit (900) can store information about the reference type of the external object predetermined by the user in the memory (220) and determine whether to set safety zones for the external objects according to the reference type. For example, operation 310 of FIG. 3 can be performed by the object recognition unit (900).

[0078] The virtual object rendering unit (910) can perform rendering processing of a visual object corresponding to a detected external object. For example, the virtual object rendering unit (910) can obtain a rendering object by performing rendering processing of the visual object. For example, the rendering object can have a shape similar to (or close to) the shape of the external object. For example, the virtual object rendering unit (910) can display the rendering object on a location within the immersive environment corresponding to the external object within the external environment. For example, the operation of the virtual object rendering unit (910) performing rendering processing of the visual object can be exemplified within the description of FIG. 7.

[0079] The control unit (920) can determine whether to set a safety zone based on the detection of external objects. For example, the control unit (920) can set a safety zone for all detected external objects, or only for some external objects. For example, the control unit (920) can determine whether to set a safety zone based on the priority of the external objects, or whether to set a safety zone based on the number of safety zones set. The control unit (920) can control the video see-through unit (940) described below.

[0080] The safety zone allocation unit (930) can set a safety zone for an external object determined by the control unit (920). The safety zone allocation unit (930) can determine the shape of the safety zone based on the shape of the external object, or expand the safety zone based on the direction of movement of the external object. For example, operation 320 of FIG. 3 can be performed by the safety zone allocation unit (930).

[0081] The video see-through unit (940) may display a visual object on the display (240) based on a change in the state of the safety zone. For example, the video see-through unit (940) may display a visual object to allow the user to view an external object through a portion of the immersive environment. For example, the video see-through unit (940) may stop (or refrain from, or not display) the visual object based on the state of the safety zone maintained for a reference period of time. For example, operation 340 of FIG. 3 may be performed by the video see-through unit (940).

[0082] FIG. 10 is a block diagram of an electronic device within a network environment according to various embodiments.

[0083] Referring to FIG. 10, in a network environment (1000), an electronic device (1001) may communicate with an electronic device (1002) via a first network (1098) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (1004) or a server (1008) via a second network (1099) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (1001) may communicate with the electronic device (1004) via the server (1008). According to one embodiment, the electronic device (1001) may include a processor (1020), a memory (1030), an input module (1050), an audio output module (1055), a display module (1060), an audio module (1070), a sensor module (1076), an interface (1077), a connection terminal (1078), a haptic module (1079), a camera module (1080), a power management module (1088), a battery (1089), a communication module (1090), a subscriber identification module (1096), or an antenna module (1097). In some embodiments, the electronic device (1001) may omit at least one of these components (e.g., the connection terminal (1078)), or may have one or more other components added. In some embodiments, some of these components (e.g., sensor module (1076), camera module (1080), or antenna module (1097)) may be integrated into a single component (e.g., display module (1060)).

[0084] The processor (1020) may, for example, execute software (e.g., a program (1040)) to control at least one other component (e.g., a hardware or software component) of the electronic device (1001) connected to the processor (1020) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (1020) may store commands or data received from other components (e.g., a sensor module (1076) or a communication module (1090)) in a volatile memory (1032), process the commands or data stored in the volatile memory (1032), and store result data in a non-volatile memory (1034). According to one embodiment, the processor (1020) may include a main processor (1021) (e.g., a central processing unit or an application processor) or an auxiliary processor (1023) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (1021). For example, when the electronic device (1001) includes the main processor (1021) and the auxiliary processor (1023), the auxiliary processor (1023) may be configured to use less power than the main processor (1021) or to be specialized for a given function. The auxiliary processor (1023) may be implemented separately from the main processor (1021) or as a part thereof.

[0085] The auxiliary processor (1023) may control at least a portion of functions or states associated with at least one component (e.g., the display module (1060), the sensor module (1076), or the communication module (1090)) of the electronic device (1001), for example, on behalf of the main processor (1021) while the main processor (1021) is in an inactive (e.g., sleep) state, or together with the main processor (1021) while the main processor (1021) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (1023) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (1080) or a communication module (1090)). In one embodiment, the auxiliary processor (1023) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (1001) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (1008)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0086] The memory (1030) can store various data used by at least one component (e.g., the processor (1020) or the sensor module (1076)) of the electronic device (1001). The data can include, for example, software (e.g., the program (1040)) and input data or output data for commands related thereto. The memory (1030) can include volatile memory (1032) or non-volatile memory (1034).

[0087] The program (1040) may be stored as software in memory (1030) and may include, for example, an operating system (1042), middleware (1044), or an application (1046).

[0088] The input module (1050) can receive commands or data to be used in a component of the electronic device (1001) (e.g., a processor (1020)) from an external source (e.g., a user) of the electronic device (1001). The input module (1050) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0089] The audio output module (1055) can output audio signals to the outside of the electronic device (1001). The audio output module (1055) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0090] The display module (1060) can visually provide information to an external party (e.g., a user) of the electronic device (1001). The display module (1060) may include, for example, a display, a holographic device, or a projector, and a control circuit for controlling the device. In one embodiment, the display module (1060) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0091] The audio module (1070) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (1070) can acquire sound through the input module (1050), output sound through the sound output module (1055), or an external electronic device (e.g., electronic device (1002)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (1001).

[0092] The sensor module (1076) can detect the operating status (e.g., power or temperature) of the electronic device (1001) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (1076) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0093] The interface (1077) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (1001) to an external electronic device (e.g., the electronic device (1002)). In one embodiment, the interface (1077) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0094] The connection terminal (1078) may include a connector through which the electronic device (1001) may be physically connected to an external electronic device (e.g., the electronic device (1002)). In one embodiment, the connection terminal (1078) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0095] The haptic module (1079) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (1079) may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0096] The camera module (1080) can capture still images and videos. In one embodiment, the camera module (1080) may include one or more lenses, image sensors, image signal processors, or flashes.

[0097] The power management module (1088) can manage power supplied to the electronic device (1001). According to one embodiment, the power management module (1088) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0098] A battery (1089) may power at least one component of the electronic device (1001). In one embodiment, the battery (1089) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0099] The communication module (1090) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (1001) and an external electronic device (e.g., electronic device (1002), electronic device (1004), or server (1008)), and the performance of communication through the established communication channel. The communication module (1090) may operate independently from the processor (1020) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (1090) may include a wireless communication module (1092) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (1094) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, a corresponding communication module can communicate with an external electronic device (1004) via a first network (1098) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (1099) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (1092) can verify or authenticate the electronic device (1001) within a communication network such as the first network (1098) or the second network (1099) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (1096).

[0100] The wireless communication module (1092) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimizing terminal power and connecting multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (1092) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (1092) may support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (1092) may support various requirements specified in the electronic device (1001), an external electronic device (e.g., the electronic device (1004)), or a network system (e.g., the second network (1099)). According to one embodiment, the wireless communication module (1092) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 104 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0101] The antenna module (1097) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (1097) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (1097) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (1098) or the second network (1099), may be selected from the plurality of antennas by, for example, the communication module (1090). A signal or power may be transmitted or received between the communication module (1090) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (1097).

[0102] According to various embodiments, the antenna module (1097) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band.

[0103] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0104] According to one embodiment, commands or data may be transmitted or received between the electronic device (1001) and an external electronic device (1004) via a server (1008) connected to a second network (1099). Each of the external electronic devices (1002 or 1004) may be the same or a different type of device as the electronic device (1001). According to one embodiment, all or part of the operations executed in the electronic device (1001) may be executed in one or more of the external electronic devices (1002, 1004, or 1008). For example, when the electronic device (1001) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (1001) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (1001). The electronic device (1001) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (1001) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (1004) may include an Internet of Things (IoT) device. The server (1008) may be an intelligent server utilizing machine learning and / or a neural network.According to one embodiment, an external electronic device (1004) or server (1008) may be included within the second network (1099). The electronic device (1001) may be applied to intelligent services (e.g., smart homes, smart cities, smart cars, or healthcare) based on 5G communication technology and IoT-related technology.

[0105] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0106] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0107] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0108] Various embodiments of the present document may be implemented as software (e.g., a program (1040)) including one or more instructions stored in a storage medium (e.g., an internal memory (1036) or an external memory (1038)) readable by a machine (e.g., an electronic device (1001)). For example, a processor (e.g., a processor (1020)) of the machine (e.g., an electronic device (1001)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0109] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0110] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0111] Figures 11a and 11b show an example of the appearance of a wearable device.

[0112] FIGS. 11A and 11B illustrate an example of an exterior appearance of a wearable device (e.g., a wearable device (200)). The wearable device (200) of FIGS. 11A and 11B may be an example of the wearable device (200) of FIG. 2. According to one embodiment, an example of an exterior appearance of a first side (1110) of a housing of a wearable device (200) may be illustrated in FIG. 11A, and an example of an exterior appearance of a second side (1120) opposite to the first side (1110) may be illustrated in FIG. 11B.

[0113] Referring to FIG. 11A, according to one embodiment, a first surface (1110) of a wearable device (200) may have a form attachable to a body part of a user (e.g., the face of the user). Although not shown, the wearable device (200) may further include a strap and / or one or more temples for being fixed to a body part of the user. A first display (1150-1) for outputting an image to a left eye among the user's two eyes, and a second display (1150-2) for outputting an image to a right eye among the user's two eyes may be disposed on the first surface (1110). The wearable device (200) is formed on the first surface (1110) and may further include a rubber or silicone packing to prevent interference by light (e.g., ambient light) different from the light emitted from the first display (1150-1) and the second display (1150-2).

[0114] According to one embodiment, the wearable device (200) may include cameras (1160-1) for photographing and / or tracking both eyes of the user adjacent to each of the first display (1150-1) and the second display (1150-2). The cameras (1160-1) may be referred to as the gaze tracking camera (1160-1) of FIG. 11B. According to one embodiment, the wearable device (200) may include cameras (1160-5, 1160-6) for photographing and / or recognizing the face of the user. The cameras (1160-5, 1160-6) may be referred to as FT cameras. The wearable device (200) may control an avatar representing the user in a virtual space based on the motion of the user's face identified using the cameras (1160-5, 1160-6). For example, the wearable device (200) may change the texture and / or shape of a portion of an avatar (e.g., a portion of an avatar expressing a human face) using information obtained by cameras (1160-5, 1160-6) (e.g., FT cameras) and representing a facial expression of a user wearing the wearable device (200).

[0115] Referring to FIG. 11b, a camera (e.g., cameras (1160-7, 1160-8, 1160-9, 1160-10, 1160-11, 1160-12)) and / or a sensor (e.g., a depth sensor (1130)) for obtaining information related to the external environment of the wearable device (200) may be disposed on a second surface (1120) opposite to the first surface (1110) of FIG. 11a. For example, the cameras (1160-7, 1160-8, 1160-9, 1160-10) may be disposed on the second surface (1120) for recognizing external objects.

[0116] For example, using cameras (1160-11, 1160-12), the wearable device (200) can obtain images and / or videos to be transmitted to each of the user's eyes. The camera (1160-11) can be placed on the second face (1120) of the wearable device (200) to obtain an image to be displayed through the second display (1150-2) corresponding to the right eye among the two eyes. The camera (1160-12) can be placed on the second face (1120) of the wearable device (200) to obtain an image to be displayed through the first display (1150-1) corresponding to the left eye among the two eyes.

[0117] According to one embodiment, the wearable device (200) may include a depth sensor (1130) disposed on the second face (1120) to identify a distance between the wearable device (200) and an external object. Using the depth sensor (1130), the wearable device (200) may obtain spatial information (e.g., a depth map) for at least a portion of the FoV of a user wearing the wearable device (200). Although not illustrated, a microphone may be disposed on the second face (1120) of the wearable device (200) to obtain a sound output from an external object. The number of microphones may be one or more depending on the embodiment.

[0118] Hereinafter, with reference to FIG. 12, the hardware or software configuration of the wearable device (200) is described.

[0119] Figure 12 shows an example of a block diagram of a wearable device.

[0120] Fig. 12 illustrates an example of a block diagram of a wearable device (e.g., a wearable device (200)). The wearable device (200) of Fig. 12 may be an example of the wearable device (200) of Fig. 2 and the wearable devices (200) of Figs. 11a to 11b.

[0121] Referring to FIG. 12, a wearable device (200) according to one embodiment may include a processor (1210), a memory (1215), a display (1150) (e.g., the first display (1150-1) and / or the second display (1150-2) of FIGS. 11A and 11B), and / or a sensor (1220). The processor (1210), the memory (1215), the display (1150), and / or the sensor (1220) may be electrically and / or operatively connected to each other by electronic components such as a communication bus (1202). In the present disclosure, the operative connection of the electronic components may include a direct connection established between the electronic components and / or an indirect connection established between the electronic components such that a first electronic component among the electronic components is controlled by a second electronic component among the electronic components. The type and / or number of electronic components included in the wearable device (200) is not limited to those illustrated in FIG. 12. For example, the wearable device (200) may include only some of the electronic components illustrated in FIG. 12.

[0122] According to one embodiment, a processor (1210) of a wearable device (200) may include a circuit (e.g., a processing circuit) for processing data based on one or more instructions. The circuit for processing data may include, for example, an arithmetic and logic unit (ALU), a field programmable gate array (FPGA), a central processing unit (CPU), and / or an application processor (AP). In one embodiment, the wearable device (200) may include one or more processors. The processor (1210) may have a multi-core processor structure such as a dual core, a quad core, a hexa core, and / or an octa core. The multi-core processor structure of the processor (1210) may include a structure (e.g., a big-little structure) based on a plurality of core circuits that are distinguished by power consumption, clock, and / or calculation amount per unit time. In one embodiment comprising a processor (1210) having a multi-core processor architecture, the operations and / or functions of the present disclosure may be performed individually or collectively by one or more cores included in the processor (1210).

[0123] The memory (1215) of the wearable device (200) according to one embodiment may include electronic components for storing data and / or instructions input to and / or output from the processor (1210). The memory (1215) may include, for example, volatile memory such as random-access memory (RAM) and / or non-volatile memory such as read-only memory (ROM). The volatile memory may include, for example, at least one of dynamic RAM (DRAM), static RAM (SRAM), cache RAM, and pseudo SRAM (PSRAM). The non-volatile memory may include, for example, at least one of programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, hard disk, compact disc, and embedded multimedia card (eMMC). In one embodiment, memory (1215) may be referred to as storage.

[0124] In one embodiment, a display (1150) of a wearable device (200) can output visualized information to a user of the wearable device (200). The display (1150), which is arranged in front of the eyes of a user wearing the wearable device (200), can be arranged on at least a portion of a housing of the wearable device (200) (e.g., the first display (1150-1) and / or the second display (1150-2) of FIGS. 11A and 11B ). For example, the display (1150) can be controlled by a processor (1210) including circuits such as a CPU, a GPU (graphics processing unit), and / or a DPU (display processing unit), to output visualized information to the user. The display (1150) can include a flexible display, a flat panel display (FPD), and / or electronic paper. The display (1150) may include a liquid crystal display (LCD), a plasma display panel (PDP), and / or one or more light emitting diodes (LEDs). The LEDs may include organic LEDs (OLEDs). Embodiments are not limited thereto, and for example, if the wearable device (200) includes a lens for transmitting external light (or ambient light), the display (1150) may include a projector (or projection assembly) for projecting light onto the lens. In one embodiment, the display (1150) may be referred to as a display panel and / or a display module. The pixels included in the display (1150) may be arranged to face either of the user's eyes when the wearable device (200) is worn by the user.For example, the display (1150) may include display areas (or active areas) corresponding to each of the user's two eyes.

[0125] In one embodiment, the sensor (1220) of the wearable device (200) may generate electrical information that may be processed by the processor (1210) and / or the memory (1215) from non-electronic information related to the wearable device (200). For example, the sensor (1220) may include a global positioning system (GPS) sensor for detecting the geographic location of the wearable device (200). In addition to the GPS method, the sensor (1220) may generate information indicating the geographic location of the wearable device (200) based on a global navigation satellite system (GNSS), such as, for example, Galileo or Beidou (compass). The above information may be stored in memory (1215), processed by processor (1210), and / or transmitted to another electronic device distinct from the wearable device (200) via communication circuitry.

[0126] According to one embodiment, one or more instructions (or commands) representing data to be processed, calculations to be performed, and / or operations to be performed by the processor (1210) of the wearable device (200) may be stored in the memory (1215) of the wearable device (200). A set of one or more instructions may be referred to as a program, firmware, an operating system, a process, a routine, a sub-routine, and / or a software application (hereinafter, “application”). For example, the wearable device (200) and / or the processor (1210) may perform at least one of the operations of FIGS. 3 to 8 when a set of a plurality of instructions distributed in the form of an operating system, firmware, a driver, a program, and / or a software application is executed. Hereinafter, the fact that a software application is installed in a wearable device (200) may mean that one or more instructions provided in the form of a software application (or package) are stored in a memory (1215), and that the one or more applications are stored in a format executable by the processor (1210) (e.g., a file having an extension specified by the operating system of the wearable device (200)). For example, an application may include a program and / or a library related to a service provided to a user.

[0127] Referring to FIG. 12, programs installed in the wearable device (200) may be included in any one of different layers, including the application layer (1240), the framework layer (1250), and / or the hardware abstraction layer (HAL) (1280), based on the target. For example, programs (e.g., modules or drivers) designed to target the hardware (e.g., the display (1150), and / or the sensor (1220)) of the wearable device (200) may be included in the hardware abstraction layer (1280). The framework layer (1250) may be referred to as an XR framework layer from the perspective of including one or more programs for providing an XR (extended reality) service. For example, the layers illustrated in FIG. 12 may be logically (or for convenience of explanation) separated, and may not mean that the address space of the memory (1215) is separated by the layers.

[0128] For example, within the framework layer (1250), programs designed to target at least one of the hardware abstraction layer (1280) and / or the application layer (1240) (e.g., a position tracker (1271), a space recognizer (1272), a gesture tracker (1273), an eye-gaze tracker (1274), and / or a face tracker (1275)) may be included. The programs included in the framework layer (1250) may provide an application programming interface (API) that is executable (or callable) based on other programs.

[0129] For example, a program designed to target users of a wearable device (200) may be included within the application layer (1240). As an example of programs included in the application layer (1240), an extended reality (XR) system user interface (UI) (1241) and / or an XR application (1242) are exemplified, but the embodiment is not limited thereto. For example, programs (e.g., software applications) included in the application layer (1240) may call an API to cause execution of functions supported by programs included in the framework layer (1250).

[0130] For example, the wearable device (200) may display one or more visual objects on the display (1150) for performing interaction with the user based on the execution of the XR system UI (1241). A visual object may refer to an object that can be placed within a screen for transmitting and / or interacting with information, such as text, an image, an icon, a video, a button, a checkbox, a radio button, a text box, a slider, and / or a table. A visual object may be referred to as a visual guide, a virtual object, a visual element, a UI element, a view object, and / or a view element. The wearable device (200) may provide the user with functions available within a virtual space based on the execution of the XR system UI (1241).

[0131] Referring to FIG. 12, a lightweight renderer (1243) and / or an XR plug-in (1244) are illustrated to be included within the XR system UI (1241), but are not limited thereto. For example, based on the XR system UI (1241), the processor (1210) may execute a lightweight renderer (1243) and / or an XR plug-in (1244) within the framework layer (1250).

[0132] For example, the wearable device (200) may acquire resources (e.g., APIs, system processes, and / or libraries) used to define, create, and / or execute a rendering pipeline that allows partial changes based on the execution of a lightweight renderer (1243). The lightweight renderer (1243) may be referred to as a lightweight render pipeline in terms of defining a rendering pipeline that allows partial changes. The lightweight renderer (1243) may include a renderer built prior to the execution of a software application (e.g., a prebuilt renderer). For example, the wearable device (200) may acquire resources (e.g., APIs, system processes, and / or libraries) used to define, create, and / or execute an entire rendering pipeline based on the execution of an XR plug-in (1244). The XR plugin (1244) can be referred to as an open XR native client from the perspective of defining (or configuring) the entire rendering pipeline.

[0133] For example, the wearable device (200) may display a screen representing at least a portion of a virtual space on the display (1150) based on the execution of the XR application (1242). The XR plug-in (1244-1) included in the XR application (1242) may include instructions that support functions similar to those of the XR plug-in (1244) of the XR system UI (1241). Descriptions of the XR plug-in (1244-1) that overlap with those of the XR plug-in (1244) may be omitted. The wearable device (200) may cause the execution of the virtual space manager (1251) based on the execution of the XR application (1242).

[0134] For example, the wearable device (200) may display an image on the display (1150) in a virtual space based on the execution of the application (1245). The application (1245) may be configured to output image information for displaying a two-dimensional image. The wearable device (200) may cause the execution of the virtual space manager (1251) based on the execution of the application (1245). The wearable device (200) may generate dual image information to display the two-dimensional image in a three-dimensional virtual space based on the execution of the application (1245). Here, the dual image information may include first image information for the left eye and second image information for the right eye, taking into account binocular disparity. In order to display the two-dimensional image in the three-dimensional virtual space, the wearable device (200) may generate the dual image information based on the image information for displaying the two-dimensional image.

[0135] According to one embodiment, the wearable device (200) may provide a virtual space service based on the execution of the virtual space manager (1251). For example, the virtual space manager (1251) may include a platform for supporting the virtual space service. Based on the execution of the virtual space manager (1251), the wearable device (200) may identify a virtual space formed based on the user's location indicated by data acquired through the sensor (1230), and may display at least a portion of the virtual space on the display (1150). The virtual space manager (1251) may be referred to as a composition presentation manager (CPM).

[0136] For example, the virtual space manager (1251) may include a runtime service (1252). As an example, the runtime service (1252) may be referred to as an OpenXR runtime module (or an OpenXR runtime program). The wearable device (200) may execute at least one of a user's pose prediction function, a frame timing function, and / or a spatial input function based on the execution of the runtime service (1252). As an example, the wearable device (200) may perform rendering for a virtual space service for the user based on the execution of the runtime service (1252). For example, a function related to a virtual space, executable by the application layer (1240), may be supported based on the execution of the runtime service (1252).

[0137] For example, the virtual space manager (1251) may include a pass-through manager (1253). Based on the execution of the pass-through manager (1253), the wearable device (200) may display an image and / or video representing an actual space acquired through an external camera on at least a portion of the screen while displaying a screen representing a virtual space on the display (1150).

[0138] For example, the virtual space manager (1251) may include an input manager (1254). The wearable device (200) may identify data (e.g., sensor data) acquired by executing one or more programs included in the recognition service layer (1270) based on the execution of the input manager (1254). The wearable device (200) may use the acquired data to identify a user input related to the wearable device (200). The user input may be related to a motion (e.g., a hand gesture), gaze, and / or speech of the user identified by a sensor (1220) (e.g., an image sensor (1230) such as an external camera). The user input may be identified based on an external electronic device connected (or paired) via a communication circuit.

[0139] For example, the perception abstract layer (1260) can be used for data exchange between the virtual space manager (1251) and the perception service layer (1270). From the perspective of being used for data exchange between the virtual space manager (1251) and the perception service layer (1270), the perception abstract layer (1260) can be referred to as an interface. For example, the perception abstract layer (1260) can be referenced as OpenPX. The perception abstract layer (1260) can be used for a perception client and a perception service.

[0140] According to one embodiment, the recognition service layer (1270) may include one or more programs for processing data acquired from the sensor (1220). The one or more programs may include at least one of a position tracker (1271), a space recognizer (1272), a gesture tracker (1273), and / or an eye tracker (1274). The type and / or number of the one or more programs included in the recognition service layer (1270) are not limited to those illustrated in FIG. 12.

[0141] For example, the wearable device (200) can identify the pose of the wearable device (200) using the sensor (1230) based on the execution of the position tracker (1271). The wearable device (200) can identify the 6 degrees of freedom pose (6 dof pose) of the wearable device (200) using data acquired using an external camera (e.g., an image sensor (1221)) and / or an IMU (e.g., a motion sensor (1222) including a gyro sensor, an acceleration sensor, and / or a geomagnetic sensor) based on the execution of the position tracker (1271). The position tracker (1271) may be referred to as a head tracking (HeT) module (or head tracker, head tracking program).

[0142] For example, the wearable device (200) may obtain information for providing a three-dimensional virtual space corresponding to the surrounding environment (e.g., external space) of the wearable device (200) (or the user of the wearable device (200)) based on the execution of the space recognizer (1272). The wearable device (200) may reproduce the surrounding environment of the wearable device (200) in three dimensions using data obtained using an external camera (e.g., an image sensor (1221)) based on the execution of the space recognizer (1272). The wearable device (200) may identify at least one of a plane, a slope, and stairs based on the surrounding environment of the wearable device (200) reproduced in three dimensions based on the execution of the space recognizer (1272). The space recognizer (1272) may be referred to as a scene understanding (SU) module (or a scene recognition program).

[0143] For example, the wearable device (200) may identify (or recognize) a pose and / or gesture of a hand of a user of the wearable device (200) based on the execution of the gesture tracker (1273). As an example, the wearable device (200) may identify a pose and / or gesture of a hand of a user using data acquired from an external camera (e.g., an image sensor (1221)) based on the execution of the gesture tracker (1273). As an example, the wearable device (200) may identify a pose and / or gesture of a hand of a user based on data (or images) acquired using an external camera based on the execution of the gesture tracker (1273). The gesture tracker (1273) may be referred to as a hand tracking (HaT) module (or hand tracking program) and / or a gesture tracking module.

[0144] For example, the wearable device (200) may identify (or track) eye movements of a user of the wearable device (200) based on the execution of the gaze tracker (1274). As an example, the wearable device (200) may identify eye movements of the user using data acquired from a gaze tracking camera (e.g., an image sensor (1221)) based on the execution of the gaze tracker (1274). The gaze tracker (1274) may be referred to as an eye tracking (ET) module (or eye tracking program) and / or a gaze tracking module.

[0145] For example, the recognition service layer (1270) of the wearable device (200) may further include a face tracker (1275) for tracking the user's face. For example, the wearable device (200) may identify (or track) the movement of the user's face and / or the user's expression based on the execution of the face tracker (1275). The wearable device (200) may estimate the user's expression based on the movement of the user's face based on the execution of the face tracker (1275). As an example, the wearable device (200) may identify the movement of the user's face and / or the user's expression based on data (e.g., images and / or videos) acquired using a camera (1225) (e.g., a camera facing at least a portion of the user's face) based on the execution of the face tracker (1275).

[0146] Referring to FIG. 12, the renderer (1290) may include instructions for rendering images in a three-dimensional virtual space. The processor (1210) executing the renderer (1290) may obtain at least one image to be at least partially displayed in the display area of ​​the display (1150) in a software application. For example, the processor (1210) executing the renderer (1290) may determine the location of the area where an application (e.g., XR application (1242), application (1245)) is to be rendered. The processor (1210) executing the renderer (1290) may generate an image of the application to be displayed on the display (1150). The renderer (1290) may synthesize images to generate a composite image to be displayed on the display (1150).

[0147] For example, the processor (1210) executing the renderer (1290) can divide the display area of ​​the display (1150) into a foveated portion (or may be referred to as the foveated area) and a peripheral portion (or may be referred to as the residual area) using the gaze position calculated using the position tracker (1271) and / or the gaze tracker (1274). For example, the processor (1210) detecting the coordinate values ​​of the gaze position can determine the portion of the display area including the coordinate values ​​as the foveated area. The DPU executing the renderer (1290) can obtain at least one image corresponding to each of the foveated area and the residual area, and having a size smaller than the size of the entire display area of ​​the display (1150) or a resolution smaller than the resolution of the display area.

[0148] The processor (1210) executing the renderer (1290) may obtain or generate a composite image to be displayed on the display (1150) by synthesizing an image corresponding to the foveated area and an image corresponding to the surrounding area. For example, the processor (1210) may perform upscaling to enlarge the image corresponding to the surrounding area to the size of the entire display area of ​​the display (1150). On the enlarged image, the processor (1210) may combine the image corresponding to the foveated area to generate a composite image to be displayed on the display (1150). Along the boundary line of the image corresponding to the foveated area, the processor (1210) may apply a visual effect, such as blur, to blend the enlarged image and the image corresponding to the foveated area.

[0149] Fig. 13 shows an example of a block diagram of an electronic device for displaying an image in a virtual space.

[0150] In Fig. 13, an example is described in which multiple programs / instructions are executed to display an image in a virtual space. The multiple programs / instructions may be executed entirely on a single processor (e.g., an AP) or by multiple processors (e.g., an AP, a GPU (graphics processing unit), or an NPU (neural processing unit)). Being able to be executed by multiple processors means that some programs / instructions may be executed by a first processor, and other programs / instructions may be executed by a second processor different from the first processor.

[0151] Referring to FIG. 13, the electronic device (1001) may execute a virtual space manager (1350) (e.g., the virtual space manager (1251) of FIG. 12, CPM) to render an image in a virtual space. For the virtual space manager (1350), at least some of the descriptions of the virtual space manager (1251) of FIG. 12 may be referenced. The virtual space manager (1350) may include a platform for supporting a virtual space service. The virtual space manager (1350) may include a runtime service (1351) (e.g., open XR runtime), a panel renderer (1352) (e.g., 2D panel render), and an XR compositor (1353). The electronic device (1001) may execute at least one of a user's pose prediction function, a frame timing function, and / or a spatial input function based on the execution of the runtime service (1351). For the runtime service (1351), at least some of the descriptions of the runtime service (1252) of FIG. 12 may be referred to. The electronic device (1001) may display at least one image (video) on a panel (e.g., a 2D panel) to implement a virtual space through the display based on the execution of the panel rendering (1352). For example, the electronic device (1001) may display a rendering image corresponding to RGB information (1366) for the panel from the spatialization manager (1340) described below through the display (e.g., the display (1150)). The electronic device (1001) may synthesize an image of an actual area captured by a camera in the virtual space (hereinafter, a pass-through image) with a virtual area image based on the execution of the XR compositor (1353). For example, the electronic device (1001) can generate a composite image by merging the pass-through image and the virtual area image based on the execution of the XR synthesis unit (1353).The electronic device (1001) may transmit the generated composite image to a display buffer so that the composite image is displayed. The electronic device (1001) may identify a virtual space through a virtual space manager (1350) and display at least a portion of the virtual space on the display (1150). The virtual space manager (1350) may be referred to as a CPM. The electronic device (1001) may execute the virtual space manager (1350) to render an image corresponding to at least a portion of the virtual space.

[0152] According to one embodiment, the electronic device (1001) may execute a spatialization manager (1340). The spatialization manager (1340) may perform processes for displaying an image in a three-dimensional virtual space. The electronic device (1001) may perform preprocessing based on the execution of the spatialization manager (1340) so that the image can be rendered in a three-dimensional virtual space through the virtual space manager (1350). For example, the electronic device (1001) may perform at least some of the functions of the renderer (1290) of FIG. 12 based on the execution of the spatialization manager (1340). The electronic device (1001) may process image information provided by an application (e.g., an XR application (1310), an application (1320) that provides a general 2D screen other than XR, and an application that provides a system UI (1330)) based on the execution of the spatialization manager (1340). A spatialization manager (1340) (e.g., space flinger) may include a system scene manager (1341) (e.g., system scene), an input manager (1342) (e.g., input routing), and a lightweight rendering engine (1343) (e.g., impress engine). The system scene manager (1341) may be executed to display a system UI (1330). System UI-related information (1364) may be transmitted to the system scene manager (1341) from a program (e.g., API) that provides the system UI (1330). The system UI-related information (1364) may be obtained through a spatializer API and / or a same-process private API. The spatialization manager (1340) may determine the layout (e.g., location, display order) of the screen of the system UI (1330) in a three-dimensional space through pre-allocated resources.The system screen manager (1341) may transmit image information (1367) for rendering the screen of the system UI (1330) to the virtual space manager (1350) according to the layout. The input manager (1342) may be configured to process user input (e.g., user input on a system screen or an app screen). The impression engine (1343) may be a renderer for image generation (e.g., a lightweight renderer (1243)). For example, the impression engine (1343) may be used to display the system UI (1330). According to one embodiment, the spatialization manager (1340) may include a lightweight rendering engine (1343) for rendering the system UI. According to one embodiment, when the lightweight rendering engine (1343) does not have sufficient resources to render an avatar used in the HMD, at least one external rendering engine may be used. At this time, to resolve compatibility issues with external rendering (e.g., 3rd party engines), an external rendering engine support module may be added within the spatialization manager (1340).

[0153] According to one embodiment, the electronic device can execute an application. For example, in response to the execution of an XR application (1310) (e.g., an XR application (1242), a 3D game, an XR map, or other immersive application), the electronic device can execute a virtual space manager (1350). The electronic device (1001) can provide dual image information (1361) provided from the XR application (1310) to the virtual space manager (1350). In order to display an image in a three-dimensional space, the dual image information (1361) can include two pieces of image information that take binocular parallax into account. For example, the dual image information (1361) can include first image information for the user's left eye and second image information for the user's right eye for rendering in a three-dimensional virtual space. Hereinafter, in the present disclosure, the term dual image information is used to refer to image information for displaying images for both eyes in a three-dimensional space. In addition to dual image information, the above dual image information may also include binocular image information, dual image information, dual image data, dual images, binocular image data, stereoscopic image information, 3D image information, spatial image information, spatial image data, 2D-3D conversion data, dimensional conversion image data, binocular parallax image data, and / or equivalent technical terms. The electronic device (1001) may generate a composite image by merging image layers through a virtual space manager (1350). The electronic device (1001) may transmit the generated composite image to a display buffer. The composite image may be displayed on the display (1150) of the electronic device (1001).

[0154] According to one embodiment, the electronic device can execute at least one application among an XR application (1310) and other applications (1320) (e.g., a first application (1320-1), a second application (1320-2), ..., an Nth application (1320-N)). According to one embodiment, the application (1320) can be configured to output image information for displaying a two-dimensional image. In other words, the application (1320) can provide a two-dimensional image. For example, the application (1320) can be a video application, a schedule application, or an application (1320) can be an Internet browser application. If it is assumed that in response to the execution of the application (1320), image information (1362) provided from the application (1320) is provided to the virtual space manager (1350). Since the image information (1362) only has x-coordinates and y-coordinates within a two-dimensional plane, it may be difficult to consider the chronological relationship (i.e., the distance from the user) between other applications centered on the user. Even when displaying an application (1320) that provides a general 2D screen, the electronic device (1001) may execute the spatialization manager (1340) to provide dual image information to the virtual space manager (1350). For example, based on the execution of the spatialization manager (1340), the electronic device (1001) may receive application-related information (1363) from the first application (1320-1). For example, the application-related information (1363) may include image information representing a two-dimensional image of the first application (1320-1) (e.g., information including RGB for each pixel) and / or content information in the first application (1320-1) (e.g., characteristics of content executed in the first application, type of content). Application related information (1363) can be obtained through the spatializer API.Based on the execution of the spatialization manager (1340), the electronic device (1001) can identify information about the location of the area to be rendered by the first application (1320-1) and the size of the area to be rendered (hereinafter, location information). Based on the execution of the spatialization manager (1340), the electronic device (1001) can generate dual image information (1365, e.g., RGBx2) that takes into account the user's binocular disparity through the image information and the location information. Based on the execution of the spatialization manager (1340), the electronic device (1001) can provide the dual image information (1365) to the virtual space manager (1350). By converting a simple two-dimensional image into the dual image information (1365), a problem that occurs when the image information (1362) is directly transmitted to the virtual space manager (1350) can be resolved. Additionally, since at least some of the functions for displaying images in a virtual space are performed by the spatialization manager (1340) instead of the virtual space manager (1350), the burden on the virtual space manager (1350) can be reduced.

[0155] The technical problems to be achieved in the present disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by a person having ordinary knowledge in the technical field to which the present disclosure pertains.

[0156] As described above, the wearable device (e.g., the wearable device (200) of FIG. 2) may include a memory (e.g., the memory (220) of FIG. 2) for storing instructions and including one or more storage media, one or more cameras (e.g., one or more cameras (230) of FIG. 2), a display (e.g., the display (240) of FIG. 2), and at least one processor (e.g., at least one processor (210) of FIG. 2) including processing circuitry. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display an immersive environment (e.g., the immersive environment (600) of FIG. 6) on the display. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to detect, through the one or more cameras, an external object (e.g., an external object (410) in FIG. 4) in front of the wearable device worn by a user. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to establish a safety zone (e.g., a safety zone (405) in FIG. 4) located within a threshold distance from the external object detected while displaying the immersive environment. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify a state of the safety zone.The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display, on the display, a visual object (e.g., visual object (605, 610) of FIG. 6) corresponding to the external object and acquired through the one or more cameras, based on a change in the state of the safety zone identified while displaying the immersive environment, to authorize the user to view the external object through a portion of the immersive environment.

[0157] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify movement of the external object via the one or more cameras. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to move the safety zone in response to the movement of the external object while displaying the immersive environment. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display the visual object based on a change in the state of the safety zone in response to the movement of the safety zone.

[0158] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify, via the one or more cameras, the direction of movement of the external object. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to expand the safety zone in the direction while displaying the immersive environment. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display the visual object based on a change in the state of the safety zone resulting from the expansion of the safety zone.

[0159] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify a speed of the external object via the one or more cameras. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to expand the safety zone in the direction proportional to the speed while displaying the immersive environment.

[0160] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to detect, via the one or more cameras, another external object in front of the wearable device moving into the safety zone. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display the visual object based on a change in the state of the safety zone as the another external object moves into the safety zone. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display, on the display, another visual object corresponding to the other external object and acquired through the one or more cameras to authorize the user to view the other external object through a portion of the immersive environment, based on a change in the state of the safety zone as the other external object moves into the safety zone.

[0161] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to detect another external object in front of the wearable device via the one or more cameras. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to establish another safety zone located within another threshold distance from the another external object, based on the another external object detected while displaying the immersive environment. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify a change in the state of the safety zone due to an overlap of at least a portion of the safety zone with at least a portion of the other safety zone. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display, on the display, an area corresponding to an extent of the safety zone acquired through the one or more cameras for authorizing the user to view the external object, based on the change in the state of the safety zone due to the overlapping of at least a portion of the safety zone with at least a portion of the other safety zone. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display, on the display, an area corresponding to an extent of the other safety zone acquired through the one or more cameras for authorizing the user to view the other external object, based on the change in the state of the safety zone due to the overlapping of at least a portion of the other safety zone with at least a portion of the other safety zone.

[0162] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to detect, via the one or more cameras, at least a portion of the user's body moving into the safety zone. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display the visual object based on a change in the state of the safety zone as the at least a portion of the user's body moves into the safety zone.

[0163] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display the visual object on the display by displaying an area corresponding to an area of ​​the safety zone and acquired through the one or more cameras, based on the change in the state of the safety zone identified while displaying the immersive environment.

[0164] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify whether the state of the safety zone is maintained for a reference period of time while displaying the visual object. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to stop displaying the visual object based on the state of the safety zone being maintained for the reference period of time.

[0165] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to obtain a rendering object by performing rendering processing on the visual object acquired through the one or more cameras based on the external object detected while displaying the immersive environment. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display the rendering object at a location within the immersive environment where the visual object is to be displayed. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display the visual object and to stop displaying the rendering object based on the change in the state of the safety zone.

[0166] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify a type of the external object via the one or more cameras. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to set the safety zone based on the type of the external object corresponding to a reference type. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to refrain from setting the safety zone based on the type of the external object that is different from the reference type.

[0167] The method described above may be performed in a wearable device comprising one or more cameras and a display. The method may include displaying an avatar representing a user on the display assembly. The method may include displaying an immersive environment on the display. The method may include detecting an external object in front of the wearable device worn by the user through the one or more cameras. The method may include establishing a safety zone located within a threshold distance from the external object based on the external object detected while displaying the immersive environment. The method may include identifying a state of the safety zone. The method may include displaying a visual object corresponding to the external object and acquired through the one or more cameras on the display based on a change in the state of the safety zone identified while displaying the immersive environment to authorize the user to view the external object through a portion of the immersive environment.

[0168] For example, the method may include an operation of identifying movement of the external object through the one or more cameras. The method may include an operation of moving the safety zone according to the movement of the external object while displaying the immersive environment. The method may include an operation of displaying the visual object based on a change in the state of the safety zone according to the movement of the safety zone.

[0169] For example, the method may include an operation of identifying the direction of movement of the external object through the one or more cameras. The method may include an operation of expanding the safety zone in the direction while displaying the immersive environment. The method may include an operation of displaying the visual object based on a change in the state of the safety zone resulting from the expansion of the safety zone.

[0170] For example, the method may include an operation of identifying the speed of the external object through the one or more cameras. The method may include an operation of expanding the safety zone in the direction proportional to the speed while displaying the immersive environment.

[0171] For example, the method may include detecting, through the one or more cameras, another external object in front of the wearable device moving into the safety zone. The method may include displaying, based on a change in the state of the safety zone as the another external object moves into the safety zone, the visual object. The method may include displaying, on the display, another visual object corresponding to the another external object and acquired through the one or more cameras, based on a change in the state of the safety zone as the another external object moves into the safety zone, to authorize the user to view the another external object through a portion of the immersive environment.

[0172] For example, the method may include detecting another external object in front of the wearable device through the one or more cameras. The method may include setting another safety zone located within another threshold distance from the other external object based on the other external object detected while displaying the immersive environment. The method may include identifying a change in the state of the safety zone due to an overlap between at least a portion of the other safety zone and at least a portion of the safety zone. The method may include displaying an area corresponding to an area of ​​the safety zone acquired through the one or more cameras to authorize the user to view the external object based on the change in the state of the safety zone due to an overlap between at least a portion of the other safety zone and at least a portion of the safety zone. The method may include an action of displaying, on the display, an area corresponding to an area of ​​the other safety zone acquired through the one or more cameras to authorize the user to view the other external object, based on the change in the state of the other safety zone due to the overlapping of at least a portion of the other safety zone and at least a portion of the other safety zone.

[0173] For example, the method may include an action of detecting, through the one or more cameras, at least a portion of the user's body moving into the safety zone. The method may include an action of displaying the visual object based on a change in the state of the safety zone as the at least a portion of the user's body moves into the safety zone.

[0174] For example, the method may include an action of displaying the visual object by displaying an area corresponding to the size of the safety zone and acquired through the one or more cameras on the display based on the change in the state of the safety zone identified while displaying the immersive environment.

[0175] For example, the method may include an operation of identifying whether the state of the safety zone is maintained for a reference time while displaying the visual object. The method may include an operation of stopping displaying the visual object based on the state of the safety zone being maintained for the reference time.

[0176] For example, the method may include an operation of obtaining a rendering object by performing a rendering process on the visual object acquired through the one or more cameras based on the external object detected while displaying the immersive environment. The method may include an operation of displaying the rendering object on a location where the visual object is to be displayed within the immersive environment. The method may include an operation of displaying the visual object and stopping displaying the rendering object based on the change in the state of the safety zone.

[0177] For example, the method may include an operation of identifying the type of the external object through the one or more cameras. The method may include an operation of setting the safety zone based on the type of the external object corresponding to a reference type. The method may include an operation of refraining from setting the safety zone based on the type of the external object that is different from the reference type.

[0178] The non-transitory computer-readable storage medium described above may store one or more programs. The one or more programs may include instructions that, when executed by a wearable device including one or more cameras and a display, cause the wearable device to display an immersive environment on the display. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to detect an external object in front of the wearable device worn by a user through the one or more cameras. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to set a safety zone located within a threshold distance from the external object based on the external object detected while displaying the immersive environment. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to identify a state of the safety zone. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to display, on the display, a visual object corresponding to the external object and acquired through the one or more cameras to authorize the user to view the external object through a portion of the immersive environment, based on a change in the state of the safety zone identified while displaying the immersive environment.

[0179] For example, the one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to identify movement of the external object via the one or more cameras. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to move the safety zone according to the movement of the external object while displaying the immersive environment. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to display the visual object based on a change in a state of the safety zone according to the movement of the safety zone.

[0180] For example, the one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to identify a direction of movement of the external object via the one or more cameras. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to expand the safety zone in the direction while displaying the immersive environment. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to display the visual object based on a change in a state of the safety zone resulting from the expansion of the safety zone.

[0181] For example, the one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to identify a speed of the external object via the one or more cameras. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to expand the safety zone in the direction proportional to the speed while displaying the immersive environment.

[0182] For example, the one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to detect, through the one or more cameras, another external object in front of the wearable device moving into the safety zone. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to display the visual object based on a change in the state of the safety zone as the another external object moves into the safety zone. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to, based on a change in the state of the safety zone as the other external object moves into the safety zone, display on the display another visual object corresponding to the other external object and acquired through the one or more cameras to authorize the user to view the other external object through a portion of the immersive environment.

[0183] For example, the one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to detect another external object in front of the wearable device via the one or more cameras. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to establish another safety zone located within another threshold distance from the other external object, based on the other external object detected while displaying the immersive environment. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to identify a change in the state of the safety zone due to at least a portion of the safety zone overlapping at least a portion of the other safety zone. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to display, on the display, an area corresponding to an extent of the safety zone acquired through the one or more cameras for authorizing the user to view the external object, based on the change in the state of the safety zone due to the overlapping of at least a portion of the other safety zone with at least a portion of the safety zone. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to display, on the display, an area corresponding to an extent of the other safety zone acquired through the one or more cameras for authorizing the user to view the other external object, based on the change in the state of the safety zone due to the overlapping of at least a portion of the other safety zone with at least a portion of the safety zone.

[0184] For example, the one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to detect, via the one or more cameras, at least a portion of the user's body moving into the safety zone. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to display the visual object based on a change in the state of the safety zone as the at least a portion of the user's body moves into the safety zone.

[0185] For example, the one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to display the visual object on the display by displaying an area corresponding to an area of ​​the safety zone and acquired through the one or more cameras, based on the change in the state of the safety zone identified while displaying the immersive environment.

[0186] For example, the one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to identify whether the state of the safety zone is maintained for a reference period of time while displaying the visual object. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to stop displaying the visual object based on the state of the safety zone being maintained for the reference period of time.

[0187] For example, the one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to obtain a rendering object by performing rendering processing on the visual object acquired through the one or more cameras based on the external object detected while displaying the immersive environment. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to display the rendering object at a location within the immersive environment where the visual object is to be displayed. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to display the visual object and to stop displaying the rendering object based on the change in the state of the safety zone.

[0188] For example, the one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to identify a type of the external object via the one or more cameras. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to set the safety zone based on the type of the external object corresponding to a reference type. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to refrain from setting the safety zone based on the type of the external object that is different from the reference type.

[0189] The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects that are not mentioned will be clearly understood by a person having ordinary skill in the art to which the present disclosure pertains.

Claims

1. In a wearable device (200), A memory (220) storing instructions and including one or more storage media; One or more cameras (230); display (240); and At least one processor (210) comprising a processing circuit, The above instructions, when executed individually or collectively by the at least one processor (210), An immersive environment (600) is displayed on the above display (240), Through one or more of the above cameras (230), an external object (410, 500, 515, 530) is detected in front of the wearable device (200) worn by the user, Based on the external objects (410, 500, 515, 530) detected while displaying the above immersive environment (600), a safety zone (405, 510, 520, 535) located within a critical distance from the external objects (410, 500, 515, 530) is set, Identify the status of the above safety zone (405, 510, 520, 535), and Based on the change in the state of the safety zone (405, 510, 520, 535) identified while displaying the immersive environment (600), displaying on the display (240) a visual object (605, 610) corresponding to the external object (410, 500, 515, 530) and acquired through the one or more cameras (230) to authorize the user to view the external object (410, 500, 515, 530) through a part of the immersive environment (600). causing the above wearable device (200), Wearable device (200).

2. In claim 1, The above instructions, when executed individually or collectively by the at least one processor (210), Through one or more of the above cameras (230), movement of the external object (410, 500, 515, 530) is identified, While displaying the above immersive environment (600), the safety zone (405, 510, 520, 535) moves according to the movement of the external object (410, 500, 515, 530), and Based on the change in the state of the safety zone (405, 510, 520, 535) according to the movement of the safety zone (405, 510, 520, 535), to display the visual object (605, 610). causing the above wearable device (200), Wearable device (200).

3. In claim 2, The above instructions, when executed individually or collectively by the at least one processor (210), Through one or more of the above cameras (230), the direction of movement (505) of the external object (410, 500, 515, 530) is identified, While displaying the above immersive environment (600), the safety zone (405, 510, 520, 535) is extended in the direction (505), and Based on the change in the state of the safety zone (405, 510, 520, 535) according to the expansion of the safety zone (405, 510, 520, 535), to display the visual object (605, 610). causing the above wearable device (200), Wearable device (200).

4. In claim 3, The above instructions, when executed individually or collectively by the at least one processor (210), Through one or more of the above cameras (230), the speed of the external object (410, 500, 515, 530) is identified, and While displaying the above immersive environment (600), the safety zone (405, 510, 520, 535) is expanded in the direction (505) in proportion to the speed. causing the above wearable device (200), Wearable device (200).

5. In claim 1, The above instructions, when executed individually or collectively by the at least one processor (210), Through one or more of the cameras (230), detecting another external object (410, 500, 515, 530) in front of the wearable device (200) moving into the safety zone (405, 510, 520, 535), and Based on the change in the state of the safety zone (405, 510, 520, 535) as the other external object (410, 500, 515, 530) moves into the safety zone (405, 510, 520, 535), display the visual object (605, 610), and display on the display (240) another visual object (605, 610) corresponding to the other external object (410, 500, 515, 530) and acquired through the one or more cameras (230) to authorize the user to view the other external object (410, 500, 515, 530) through a part of the immersive environment (600). causing the above wearable device (200), Wearable device (200).

6. In claim 1, The above instructions, when executed individually or collectively by the at least one processor (210), Through one or more of the above cameras (230), detecting another external object (410, 500, 515, 530) in front of the wearable device (200), Based on the other external objects (410, 500, 515, 530) detected while displaying the above immersive environment (600), another safety zone (405, 510, 520, 535) located within another threshold distance from the other external objects (410, 500, 515, 530) is set, Identifying a change in the state of the safety zone (405, 510, 520, 535) due to at least a portion of the other safety zone (405, 510, 520, 535) overlapping with at least a portion of the safety zone (405, 510, 520, 535), and Based on the change in the state of the safety zone (405, 510, 520, 535) due to the overlapping of at least a portion of the other safety zone (405, 510, 520, 535) and at least a portion of the safety zone (405, 510, 520, 535), an area corresponding to the width of the safety zone (405, 510, 520, 535) acquired through the one or more cameras (230) is displayed to authorize the user to view the external object (410, 500, 515, 530), and on the display (240), the other external object (410, 500, 515, 530) acquired through the one or more cameras (230) is displayed to authorize the user to view the other external object (410, 500, 515, 530). To display an area corresponding to the width of the safety zone (405, 510, 520, 535), causing the above wearable device (200), Wearable device (200).

7. In claim 1, The above instructions, when executed individually or collectively by the at least one processor (210), Through one or more of the cameras (230), detecting at least a part of the user's body moving into the safety zone (405, 510, 520, 535), and To display the visual object (605, 610) based on a change in the state of the safety zone (405, 510, 520, 535) as at least a part of the body of the user moves into the safety zone (405, 510, 520, 535). causing the above wearable device (200), Wearable device (200).

8. In claim 1, The above instructions, when executed individually or collectively by the at least one processor (210), Based on the change in the state of the safety zone (405, 510, 520, 535) identified while displaying the immersive environment (600), displaying the visual object (605, 610) by displaying an area corresponding to the width of the safety zone (405, 510, 520, 535) and acquired through the one or more cameras (230) on the display (240). causing the above wearable device (200), Wearable device (200).

9. In claim 1, The above instructions, when executed individually or collectively by the at least one processor (210), While displaying the above visual object (605, 610), identify whether the state of the above safety zone (405, 510, 520, 535) is maintained for a reference time, and Based on the state of the safety zone (405, 510, 520, 535) maintained for the above reference time, to stop displaying the visual object (605, 610). causing the above wearable device (200), Wearable device (200).

10. In claim 1, The above instructions, when executed individually or collectively by the at least one processor (210), While displaying the immersive environment (600), based on the detected external objects (410, 500, 515, 530), a rendering object (700) is obtained by performing rendering processing on the visual objects (605, 610) obtained through the one or more cameras (230), Displaying the above rendering object on the location where the visual object (605, 610) is to be displayed in the above immersive environment (600), and Based on the change in the state of the safety zone (405, 510, 520, 535), display the visual object (605, 610) and stop displaying the rendering object (700). causing the above wearable device (200), Wearable device (200).

11. In claim 1, The above instructions, when executed individually or collectively by the at least one processor (210), Through one or more of the above cameras (230), the type of the external object (410, 500, 515, 530) is identified, Based on the type of the external object (410, 500, 515, 530) corresponding to the reference type, the safety zone (405, 510, 520, 535) is set, and Based on the type of the external object (410, 500, 515, 530) different from the above reference type, avoid setting the safety zone (405, 510, 520, 535). causing the above wearable device (200), Wearable device (200).

12. A method executed within a wearable device (200) including one or more cameras (230) and a display (240), the method comprising: An operation of displaying an immersive environment (600) on the above display (240), An operation of detecting an external object (410, 500, 515, 530) in front of the wearable device (200) worn by the user through one or more of the cameras (230), An operation of setting a safety zone (405, 510, 520, 535) located within a threshold distance from the external object (410, 500, 515, 530) based on the external object (410, 500, 515, 530) detected while displaying the immersive environment (600), An operation for identifying the status of the above safety zone (405, 510, 520, 535), and An operation including displaying, on the display (240), a visual object (605, 610) corresponding to the external object (410, 500, 515, 530) and acquired through the one or more cameras (230) based on a change in the state of the safety zone (405, 510, 520, 535) identified while displaying the immersive environment (600), to authorize the user to view the external object (410, 500, 515, 530) through a part of the immersive environment (600). method.

13. In claim 12, the method comprises: An operation of identifying movement of the external object (410, 500, 515, 530) through one or more of the cameras (230), While displaying the above immersive environment (600), an action of moving the safety zone (405, 510, 520, 535) according to the movement of the external object (410, 500, 515, 530), and An operation of displaying the visual object (605, 610) based on a change in the state of the safety zone (405, 510, 520, 535) according to the movement of the safety zone (405, 510, 520, 535), method.

14. In claim 13, the method comprises: An operation of identifying the direction (505) of movement of the external object (410, 500, 515, 530) through one or more of the cameras (230), While displaying the above immersive environment (600), an operation of expanding the safety zone (405, 510, 520, 535) in the above direction (505), and An operation of displaying the visual object (605, 610) based on a change in the state of the safety zone (405, 510, 520, 535) according to the expansion of the safety zone (405, 510, 520, 535), method.

15. In claim 14, the method comprises: An operation of identifying the speed of the external object (410, 500, 515, 530) through one or more of the cameras (230), and While displaying the above immersive environment (600), including an action of expanding the safety zone (405, 510, 520, 535) in the above direction in proportion to the speed, method.

Images