Head-mounted electronic device, method, and non-transitory computer-readable storage medium for touch input in three-dimensional space

Abstract

This head-mounted electronic device may comprise: at least one processor including a processing circuit; a display assembly including a display; and a memory for storing one or more programs configured to be individually or collectively executed by the at least one processor and including one or more storage media, wherein the one or more programs include instructions which cause the head-mounted electronic device to: while displaying a virtual object in a three-dimensional (3D) space provided through the display assembly, enter a touch input mode in which a user's hand that is in contact with a user interface (UI) object is recognized as a user input; identify first depth data of the virtual object on the basis of the entry into the touch input mode; and on the basis of identifying that the first depth data of the virtual object is out of a reference depth range, adjust the first depth data of the virtual object to second depth data in the reference depth range, so as to change a display position of the virtual object.

Description

Head-worn electronic device, method, and non-transitory computer-readable storage medium for touch input within 3D space

[0001] The present disclosure relates to a head-worn electronic device, method, and non-transitory computer-readable storage medium for touch input within 3D space.

[0002] To provide an enhanced user experience, electronic devices are being developed that provide augmented reality (AR) services that display computer-generated information in conjunction with external objects in the real world. These electronic devices may be head-mounted electronic devices that can be worn by a user. These electronic devices may be AR glasses and / or head-mounted devices (HMDs).

[0003] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art related to the present disclosure.

[0004] According to one embodiment, a head-worn electronic device is described. The head-worn electronic device may include at least one processor including a processing circuit, a display assembly, and a memory storing one or more programs configured to be individually or collectively executed by the at least one processor, the memory including one or more storage media. The one or more programs may include instructions causing the head-worn electronic device to display a virtual object in a three-dimensional (3D) space provided through the display assembly. The one or more programs may include instructions causing the head-worn electronic device to enter a touch input mode for recognizing a user's hand in contact with a user interface (UI) object as a user input while displaying the virtual object in the 3D space. The one or more programs may include instructions causing the head-worn electronic device to identify first depth data of the virtual object based on entering the touch input mode. The one or more programs may include instructions that cause the head-mounted electronic device to change a display position of the virtual object by adjusting the first depth data of the virtual object to second depth data within the reference depth range based on identifying that the first depth data of the virtual object is outside a reference depth range.

[0005] According to one embodiment, a method is described. The method may be performed in a head-mounted electronic device including a display assembly. The method may include an operation of displaying a virtual object within a three-dimensional (3D) space provided through the display assembly. The method may include an operation of entering a touch input mode for recognizing a user's hand in contact with a user interface (UI) object as a user input while displaying the virtual object within the 3D space. The method may include an operation of identifying first depth data of the virtual object based on entering the touch input mode. The method may include an operation of changing a display position of the virtual object by adjusting the first depth data of the virtual object to second depth data within the reference depth range based on identifying that the first depth data of the virtual object is outside a reference depth range.

[0006] According to one embodiment, a non-transitory computer-readable storage medium is described. The non-transitory computer-readable storage medium may store one or more programs. The one or more programs may include instructions that, when executed by a head-worn electronic device including a display assembly, cause the head-worn electronic device to display a virtual object within a three-dimensional (3D) space provided through the display assembly. The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to enter a touch input mode that recognizes a user's hand in contact with a user interface (UI) object as a user input while displaying the virtual object within the 3D space. The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to identify first depth data of the virtual object based on entering the touch input mode. The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to change a display position of the virtual object by adjusting the first depth data of the virtual object to second depth data within the reference depth range based on identifying that the first depth data of the virtual object is outside a reference depth range.

[0007] Furthermore, the above and other aspects, features, and advantages of specific embodiments of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0008] FIG. 1 is a diagram illustrating examples of errors in performing touch input on a virtual object within a virtual 3D space according to various embodiments;

[0009] FIG. 2 is a block diagram illustrating an exemplary configuration of a head-worn electronic device according to various embodiments;

[0010] FIG. 3 is a flowchart illustrating exemplary operations of a head-worn electronic device for identifying first depth data of a virtual object according to various embodiments;

[0011] FIG. 4 is a flowchart illustrating exemplary operations of a head-worn electronic device according to various embodiments depending on whether first depth data of a virtual object is within a reference depth range;

[0012] FIG. 5 is a diagram illustrating an example of whether the first depth data of a virtual object according to various embodiments is within a reference depth range;

[0013] FIG. 6 is a diagram illustrating an example of adjusting a first size of a virtual object to a second size within a reference size range according to various embodiments;

[0014] FIG. 7 is a flowchart illustrating exemplary operations of a head-mounted electronic device for comparing second depth data of an external object with third depth data within a reference depth range according to various embodiments;

[0015] FIG. 8 is a diagram illustrating examples of second depth data of an external object smaller than a reference depth range and second depth data of an external object larger than a reference depth range according to various embodiments;

[0016] FIG. 9 is a diagram illustrating an example of changing the display position of a virtual object according to various embodiments;

[0017] FIG. 10 is a flowchart illustrating exemplary operations of a head-worn electronic device for comparing second depth data of an external object with reference depth data according to various embodiments;

[0018] FIG. 11 is a diagram illustrating an example of changing the display position of a virtual object by comparing second depth data of an external object with reference depth data according to various embodiments;

[0019] FIG. 12 is a diagram illustrating an example of changing the display positions of multiple virtual objects according to various embodiments;

[0020] FIG. 13 illustrates an example of maintaining the display position of a virtual object according to changes in the movement of the user and the direction of the user's head according to various embodiments;

[0021] FIG. 14 is a flowchart illustrating exemplary operations of a head-worn electronic device for re-changing the display position of a virtual object according to various embodiments;

[0022] FIG. 15 is a diagram illustrating an example of changing the size and display position of a virtual object according to various embodiments;

[0023] FIG. 16 is a block diagram illustrating examples of configurations of head-worn electronic devices according to various embodiments.

[0024] FIG. 17 is a block diagram of an electronic device within a network environment according to various embodiments.

[0025] Hereinafter, various embodiments of the present disclosure will be described in more detail with reference to the drawings. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0026] FIG. 1 is a diagram illustrating examples of errors in performing touch input on a virtual object within a virtual 3D space according to various embodiments.

[0027] Referring to FIG. 1, the head-mounted electronic device (100) may include a head-mounted display (HMD) wearable on the head of a user (110). For example, and without limitation, the head-mounted electronic device (100) may include a head-mounted display (HMD) device, a headgear electronic device, an eyeglass-type (or goggle-type) electronic device, a video see-through (VST) device, an extended reality (XR) device, a virtual reality (VR) device, and / or an augmented reality (AR) device.

[0028] A head-worn electronic device (100) may include a display assembly (e.g., the display assembly (240) of FIG. 2). The head-worn electronic device (100) may provide a virtual three-dimensional (3D) space (115) through the display assembly. The head-worn electronic device (100) may display a virtual object (120) (or a UI object, or a visual object) within the virtual 3D space (115). The head-worn electronic device (100) may receive an input for the virtual object (120).

[0029] The head-worn electronic device (100) can receive input to the virtual object (120) based on various methods. The head-worn electronic device (100) can receive input to the virtual object (120) based on a user gesture (e.g., a pinch gesture) to the virtual object (120). The user gesture can be performed while the hand of the user (110) is away from the virtual object (120). The input to the virtual object (120) based on the user gesture can be received while the hand of the user (110) is away from the virtual object (120), but it may be required to perform multiple tracking (e.g., hand tracking, eye tracking, and / or controller tracking) to identify the user gesture. Since the user gesture is identified based on the above multiple tracking, the accuracy of the input to the virtual object (120) based on the user gesture may be relatively low.

[0030] An input to a virtual object (120) based on a user gesture may not be intuitive to the user (110). An input to a virtual object (120) based on a user gesture that is not intuitive to the user (110) may have relatively low accuracy and may cause fatigue to the user. In order to resolve this problem of an input to a virtual object (120) based on a user gesture, a head-mounted electronic device (100) may receive an input to a virtual object (120) based on a method of recognizing a hand of a user (110) in contact with a virtual object (120) as an input to the virtual object (120). An input to a virtual object (120) based on a hand of a user (110) in contact with a virtual object (120) may be defined as a touch input to the virtual object (120).

[0031] State (105) and state (125) can be described as states in which there is an error in receiving a touch input for a virtual object (120). In state (105), the head-worn electronic device (100) can display the virtual object (120) at a location relatively far away from the user (110) within the virtual 3D space (115). In state (105), the user (110) may not move in the direction of the virtual object (120) and may not perform a touch input for the virtual object (120). In order for the head-worn electronic device (100) to receive a touch input for the virtual object (120), the user (110) may be required to move in the direction of the virtual object (120). The user (110) may feel discomfort by moving in the direction of the virtual object (120) in order to perform a touch input for the virtual object (120).

[0032] Within the state (125), the head-worn electronic device (100) may display a virtual object (120) within a virtual 3D space (115). An external object (130) may be positioned between the user (110) and the virtual object (120). The external object (130) may be positioned within a real environment distinct from the virtual 3D space (115). The head-worn electronic device (100) may have an error in receiving a touch input to the virtual object (120) by the external object (130) positioned within the real environment. The user (110) may feel uncomfortable performing a touch input to the virtual object (120) by the external object (130) positioned within the real environment.

[0033] A method may be required to resolve the inconvenience of touch input for such virtual objects (120). To resolve this inconvenience, the head-mounted electronic device (100) may change the display position of the virtual object (120). To change the display position of the virtual object (120), depth data of the virtual object (120) and depth data of an external object (130) may be utilized.

[0034] The head-worn electronic device (100) may perform operations illustrated and described in more detail below with reference to FIGS. 3 to 15 to change the display position of a virtual object (120). The head-worn electronic device (100) may include components for performing the above operations. The components may be illustrated and described in more detail below with reference to FIG. 2.

[0035] FIG. 2 is a block diagram illustrating an exemplary configuration of an exemplary head-worn electronic device according to various embodiments.

[0036] Referring to FIG. 2, the head-worn electronic device (200) may be a head-mounted display (HMD) device, a headgear electronic device, a glasses-type (or goggle-type) electronic device, a video see-through (VST) device, an extended reality (XR) device, a virtual reality (VR) device, and / or an augmented reality (AR) device, or may be described as such. The head-worn electronic device (200) may include at least a portion of the electronic device (1701) of FIG. 17, or may correspond to at least a portion of the electronic device (1701) of FIG. 17. The head-worn electronic device (200) may include at least one processor (210) (e.g., including a processing circuit), a memory (220), one or more cameras (230), and a display assembly (240) (e.g., including a display).

[0037] According to one embodiment, at least one processor (210) may include various processing circuits. At least one processor (210) may include a central processing unit (CPU) (e.g., including processing circuits). At least one processor (210) may include a graphic processing unit (GPU) (e.g., including processing circuits) and a neural processing unit (NPU) (e.g., including processing circuits). At least one processor (210) may be configured to control a memory (220), one or more cameras (230), and a display assembly (240). At least one processor (210) may be configured to individually or collectively execute instructions stored in the memory (220) to cause the head-worn electronic device (200) (or the head-worn electronic device (100)) to perform at least some of the operations illustrated and described with reference to FIG. 1. At least one processor (210) may be configured to individually or collectively execute instructions stored within the memory (220) to cause the head-worn electronic device (200) to perform at least some of the operations illustrated and described in more detail below with reference to FIGS. 3 through 15.

[0038] According to one embodiment, the memory (220) may include one or more storage media. The memory (220) may store various data used by at least one component of the head-mounted electronic device (200) (e.g., at least one processor (210), the memory (220), one or more cameras (230), and / or the display assembly (240)). The data may include input data or output data for software and commands related thereto. The memory (220) may include volatile memory or non-volatile memory.

[0039] According to one embodiment, the one or more cameras (230) may include one or more optical sensors (e.g., a charged coupled device (CCD) sensor and / or a complementary metal oxide semiconductor (CMOS) sensor) that generate an electrical signal representing the color and / or brightness of light. The one or more cameras (230) may be described as an image sensor. The one or more cameras (230) may be available to acquire images of the space in front of the head-mounted electronic device (200) (or the surrounding environment). At least some of the one or more cameras (230) may have a field of view (FOV) corresponding to a field of view (FOV) of a user's eye. The FOV of some of the one or more cameras (230) may be different from the FOV of other parts of the one or more cameras (230).

[0040] According to one embodiment, the display assembly (240) may be configured to visualize information (or signals) provided from at least one processor (210). The display assembly (240) may be positioned to face the eyes of a user wearing the head-mounted electronic device (200). The display assembly (240) may be configured to provide a virtual 3D space. The display assembly (240) may be configured to display a virtual object within the virtual 3D space. The display assembly (240) may include at least one display.

[0041] The head-worn electronic device (200) illustrated within the description of FIG. 2 may execute at least some of the operations illustrated and described in more detail below with reference to FIGS. 3 through 15. The operations illustrated within the description of FIGS. 3 through 15 may be caused by (or within) the head-worn electronic device (200) under the control of at least one processor (210).

[0042] FIG. 3 is a flowchart illustrating exemplary operations of a head-worn electronic device for identifying first depth data of a virtual object according to various embodiments.

[0043] Referring to FIG. 3, in operation 300, at least one processor (210) may provide a virtual three-dimensional (3D) space (e.g., the virtual 3D space (115) of FIG. 1) through a display assembly (240). The at least one processor (210) may display a virtual object (e.g., the virtual object (120) of FIG. 1) within the virtual 3D space. The virtual object may include a user interface (UI) object and / or a window. The virtual object may be provided from a software application running within the head-mounted electronic device (200). The virtual object may include executable objects. While the virtual object is displayed within the virtual 3D space, the following operations (operations 310 and 320) may be performed.

[0044] In operation 310, according to one embodiment, at least one processor (210) may enter a touch input mode that recognizes a hand of a user (e.g., the user (110) of FIG. 1) in contact with a user interface (UI) object as a user input while displaying a virtual object in a virtual 3D space. The touch input mode may be defined as a direct touch input mode. The at least one processor (210) may recognize the hand of the user in contact with the virtual object as a touch input to the virtual object within the touch input mode. The at least one processor (210) may identify that the hand of the user is in contact with the virtual object through one or more cameras (230). The at least one processor (210) may provide a function mapped to the virtual object based on the hand of the user in contact with the virtual object within the touch input mode.

[0045] According to one embodiment, the touch input mode may be distinguished from other input modes that receive input to the virtual object by methods other than touch input. At least one processor (210) may receive input to the virtual object based on a user gesture (e.g., a pinch gesture) performed while the user's hand is away from the virtual object within the other input mode. At least one processor (210) may provide a function mapped to the virtual object based on a user gesture made to the virtual object within the other input mode.

[0046] According to one embodiment, at least one processor (210) may enter the touch input mode based on a user input and / or an event. As a non-limiting example, the user input for entering the touch input mode may include an input to a virtual object (or a virtual button) within a virtual 3D space. The at least one processor (210) may enter the touch input mode based on switching to the touch input mode from another input mode. The at least one processor (210) may enter the touch input mode for some virtual objects among a plurality of virtual objects displayed within the virtual 3D space. Upon entering the touch input mode for some of the virtual objects, the at least one processor (210) may receive a touch input for the some of the virtual objects, and may receive an input for the remaining virtual objects among the plurality of virtual objects based on a user gesture.

[0047] In operation 320, according to one embodiment, at least one processor (210) may identify first depth data of a virtual object (e.g., first depth data (515) of FIG. 5) based on entering the touch input mode. As a non-limiting example, when a location within a virtual 3D space is defined by an x-axis coordinate, a y-axis coordinate, and a z-axis coordinate, the depth data of the virtual object may represent a z-axis coordinate of the virtual object. As a non-limiting example, the depth data may represent a z-axis coordinate of a representative location within an area or space where the virtual object is displayed. By identifying the depth data of the virtual object, the at least one processor (210) may identify a distance from the user to the virtual object.

[0048] According to one embodiment, at least one processor (210) may determine whether to maintain the display position of the virtual object based on the first depth data of the identified virtual object. The use of the first depth data of the virtual object to determine whether to maintain the display position of the virtual object is further illustrated and described below with reference to FIG. 4.

[0049] FIG. 4 is a flowchart illustrating exemplary operations of a head-worn electronic device according to various embodiments depending on whether first depth data of a virtual object is within a reference depth range.

[0050] Referring to FIG. 4, according to one embodiment, in operation 400, at least one processor (210) may identify first depth data of a virtual object based on entering a touch input mode. Operation 400 may correspond to operation 320 of FIG. 3.

[0051] According to one embodiment, in operation 410, at least one processor (210) can identify whether the first depth data of the identified virtual object is within a reference depth range (e.g., the reference depth range (520) of FIG. 5). For example, the reference depth range may be referred to as a range of depth data within which the user's hand can be positioned without the user moving. The reference depth range may be predetermined or set (or changed) by the user. As a non-limiting example, the reference depth range may be set according to depth data of the user's wrist when the user extends his hand in a frontal direction. However, the disclosure is not limited thereto. Whether the first depth data of the virtual object is within the reference depth range is illustrated and described in more detail below with reference to FIG. 5.

[0052] According to one embodiment, in operation 420, at least one processor (210) can maintain the display position of the virtual object by maintaining the first depth data of the virtual object based on identifying that the first depth data (e.g., the first depth data (515) of FIG. 5) of the virtual object (e.g., the virtual object (510) of FIG. 5) is within a reference depth range (e.g., the reference depth range (520) of FIG. 5). The virtual object displayed according to the first depth data within the reference depth range within the virtual 3D space can receive a touch input from the user without the user moving (or without the user bending his or her arm). Since performing a touch input on the virtual object displayed according to the first depth data within the reference depth range within the virtual 3D space does not cause inconvenience to the user, changing the display position of the virtual object may not be required.

[0053] According to one embodiment, in operation 430, at least one processor (210) can identify a first size of the virtual object based on identifying that the first depth data of the virtual object is outside a reference depth range. The at least one processor (210) can adjust the first size of the virtual object to a second size within the reference size range. The at least one processor (210) can identify an aspect ratio of the virtual object. The at least one processor (210) can adjust the first size of the virtual object to the second size while maintaining the aspect ratio of the identified virtual object. Adjusting the size of the virtual object is illustrated and described in more detail below with reference to FIG. 6.

[0054] According to one embodiment, at operation 440, at least one processor (210) may identify, using one or more cameras (230), whether an external object (e.g., an external object (805) of FIG. 8) is positioned based on depth data that is less than a reference depth range. For example, if an external object is positioned based on depth data that is less than a reference depth range, receiving a touch input for a virtual object by the external object may have an error. At least one processor (210) may acquire images of a space in front of the head-worn electronic device (200) via one or more cameras (230). For example, at least one processor (210) may identify, using at least some of the images including the external object, whether the external object is positioned based on depth data that is less than a reference depth range. According to one embodiment, at operation 450, at least one processor (210) may identify second depth data of the external object based on the external object being positioned based on depth data that is less than the reference depth data. At least one processor (210) can acquire images of a space in front of a head-worn electronic device (200) through one or more cameras (230). At least one processor (210) can identify second depth data of an external object using at least some of the images including the external object. At least one processor (210) can identify second depth data of an external object located in a frontal direction of a user to change a display position of a window to a frontal direction of the user. The external object can be described as an external object located in a frontal direction of a user of the head-worn electronic device (200). As a non-limiting example, at least one processor (210) can identify depth values ​​of respective pixels of the images and identify second depth data of the external object using the depth values.

[0055] According to one embodiment, to alleviate user discomfort in touch input caused by the external object, second depth data of the external object may be utilized. At least one processor (210) may change the display position of a window based on the second depth data of the external object. Changing the display position of a window based on the second depth data of the external object is illustrated and described in more detail below with reference to FIG. 7.

[0056] According to one embodiment, in operation 460, at least one processor (210) may adjust the first depth data of the window to third depth data within the reference depth range based on whether an external object is not positioned according to depth data smaller than the reference depth data. For example, the at least one processor (210) may change the display position of the window by adjusting the first depth data of the window to the third depth data. For example, the at least one processor (210) may display the window according to the third depth data within the virtual 3D space. Changing the display position of the window by adjusting the first depth data of the window to the third depth data is illustrated and described in more detail below with reference to FIG. 9.

[0057] FIG. 5 is a diagram illustrating an example of whether the first depth data of a virtual object according to various embodiments is within a reference depth range.

[0058] Referring to FIG. 5, according to one embodiment, at least one processor (210) can identify first depth data (515) of a virtual object (510) displayed within a virtual 3D space (505). At least one processor (210) can identify whether the identified first depth data (515) is within a reference depth range (520).

[0059] According to one embodiment, within the state (500), at least one processor (210) may identify that the first depth data (515) of the virtual object (510) is within a reference depth range (520). Since the first depth data (515) is within the reference depth range (520), the virtual object (510) may be positioned within an area where it can receive a touch input without movement of the user. Since the virtual object (510) is positioned within an area where it can receive a touch input without movement of the user, changing the display position of the virtual object (510) may not be required. Based on identifying that the first depth data (515) is within the reference depth range (520), the at least one processor (210) may perform operation 420 of FIG. 4.

[0060] According to one embodiment, within the state (525), at least one processor (210) may identify that the first depth data (515) of the virtual object (510) is outside a reference depth range (520). Because the first depth data (515) is outside the reference depth range (520), the virtual object (510) may be relatively close to the user (501) or relatively far away. As the virtual object (510) is relatively close to the user (501), the user (501) may be required to bend an arm (or wrist) to perform a touch input to the virtual object (510). As the virtual object (510) is relatively far away from the user (501), the user (501) may be required to move in a direction toward the virtual object (510) to perform a touch input to the virtual object (510). At least one processor (210) may change the display position of the virtual object (510) to resolve user (501) discomfort caused by performing a touch input on the virtual object (510) within the state (525). At least one processor (210) may perform operations 430 and 440 of FIG. 4 based on identifying that the first depth data (515) is outside the reference depth range (520).

[0061] FIG. 6 is a diagram illustrating an example of adjusting a first size of a virtual object to a second size within a reference size range according to various embodiments.

[0062] Referring to FIG. 6, according to one embodiment, at least one processor (210) may identify a first size of the virtual object (510) and / or an aspect ratio (W:H) of the virtual object (510) based on identifying that the first depth data of the virtual object (510) is outside a reference depth range. At least one processor (210) may identify a size at which the virtual object (510) is to be rendered based on entering a touch input mode. The virtual object (510) may have a relatively large (or relatively small) first size because it is displayed according to the first depth data that is outside the reference depth range. Adjusting the relatively large (or relatively small) first size of the virtual object (510) may be required to display the virtual object (510) according to the depth data within the reference depth range.

[0063] According to one embodiment, at least one processor (210) can adjust the first size of the virtual object (510) to a second size within a reference size range (600). For example, the reference size range (600) may be referred to as a size range of the virtual object (605) set for a user to perform a touch input on the virtual object (605) when displaying the virtual object (605) according to depth data within the reference depth range. The reference size range (600) may be predetermined (e.g., specified) or set (or changed) by the user. The reference size range (600) may be composed of a reference height value (e.g., 30 cm) and a reference width value (e.g., 30 cm).

[0064] According to one embodiment, at least one processor (210) can determine a second size of a virtual object (605) based on an aspect ratio (W:H) of the virtual object (510) and / or a reference size range (600). The virtual object (605) having the second size can have an aspect ratio (W:H) corresponding to the aspect ratio (W:H) of the virtual object (510) having the first size. The at least one processor (210) can adjust the first size of the virtual object (510) to the second size while maintaining the aspect ratio (W:H) of the virtual object (510). The second size can be determined as a maximum size within the reference size range (600) in which the aspect ratio (W:H) can be maintained. The width value (W) of the virtual object (605) having the second size can be determined based on a smaller value among the width value and the height value of the reference size. As a non-limiting example, the width value (W) of the virtual object (605) having the second size may correspond to the reference width value of the reference size range (600), and the height value (H) of the virtual object (605) having the second size may be smaller than the reference height value of the reference size range (600). However, the present disclosure is not limited thereto. At least one processor (210) may change the size of the virtual object (510) by maintaining the aspect ratio (W:H) of the virtual object (510) and adjusting the first size of the virtual object (510) to the second size. At least one processor (210) may store the first size to adjust the second size of the virtual object (510) back to the first size.

[0065] FIG. 7 is a flowchart illustrating exemplary operations of a head-mounted electronic device for comparing second depth data of an external object with third depth data within a reference depth range according to various embodiments.

[0066] Referring to FIG. 7, according to one embodiment, in operation 700, at least one processor (210) may identify second depth data of an external object using one or more cameras (230). Operation 700 may correspond to operation 440 of FIG. 4.

[0067] According to one embodiment, in operation 710, at least one processor (210) can compare the second depth data of the external object with third depth data within a reference depth range. By comparing the second depth data of the external object with the reference depth range, the at least one processor (210) can identify whether the second depth data is greater than the reference depth range. Whether the second depth data of the external object is greater than the reference depth range is illustrated and described in more detail below with reference to FIG. 8.

[0068] According to one embodiment, at operation 720, the at least one processor (210) can adjust the first depth data of the window to third depth data (the third depth data (910) of FIG. 9) within the reference depth range based on identifying that the second depth data (e.g., the second depth data (810) of FIG. 8) of the external object (e.g., the external object (805) of FIG. 8) is greater than the reference depth range (e.g., the third depth data (520) of FIG. 8). The at least one processor (210) can change the display position of the window by adjusting the first depth data of the window to the third depth data. For example, the at least one processor (210) can display the window according to the third depth data within the virtual 3D space. Changing the display position of the window by adjusting the first depth data of the window to the third depth data is illustrated and described in more detail below with reference to FIG. 9.

[0069] According to one embodiment, in operation 730, at least one processor (210) may compare the second depth data of the external object with reference depth data (e.g., reference depth data (1105) of FIG. 11) based on identifying that the second depth data of the external object is smaller than the third depth data within the reference depth range. The reference depth data may be defined as the minimum depth data at which a user can perform a touch input without movement. For example, the reference depth data may be predetermined (e.g., specified) or set (or changed) by the user. As a non-limiting example, the reference depth data may correspond to the length of a user's hand. However, the present disclosure is not limited thereto.

[0070] According to one embodiment, at least one processor (210) can change the display position of a window by comparing the second depth data of the external object with the reference depth data. Changing the display position of a window by comparing the second depth data of the external object with the reference depth data is illustrated and described in more detail below with reference to FIG. 10.

[0071] FIG. 8 is a diagram illustrating examples of second depth data of an external object smaller than a reference depth range and second depth data of an external object larger than a reference depth range according to various embodiments.

[0072] Referring to FIG. 8, according to one embodiment, at least one processor (210) may identify second depth data (810) of an external object (805) using one or more cameras (230). The at least one processor (210) may identify whether the second depth data (810) is greater than the reference depth range (520) by comparing the second depth data (810) with a reference depth range (520). The at least one processor (210) may compare the second depth data (810) with the reference depth range (520) to identify whether the external object (805) is positioned closer to the user than the location where the visual object is to be displayed. If the external object (805) is positioned closer to the user than the location where the visual object is to be displayed, receiving a touch input of a virtual object by the external object (805) may have an error.

[0073] According to one embodiment, within the state (800), at least one processor (210) can identify that the second depth data (810) of the external object (805) is less than the reference depth range (520). As the second depth data (810) of the external object (805) is less than the reference depth range (520), receiving a touch input for a virtual object to be displayed according to the depth data within the reference depth range (520) may have an error by the external object (805). As receiving a touch input for a virtual object to be displayed according to the depth data within the reference depth range (520) has an error by the external object (805), displaying the virtual object according to the depth data that is less than the second depth data (810) of the external object (805) may be required. At least one processor (210) may perform operation 730 of FIG. 7 based on identifying that the second depth data (810) of the external object (805) is less than the reference depth range (520).

[0074] According to one embodiment, within the state (820), at least one processor (210) can identify that the second depth data (810) of the external object (805) is greater than the reference depth range (520). As the second depth data (810) of the external object (805) is greater than the reference depth range (520), the at least one processor (210) can receive a touch input for a virtual object to be displayed according to the depth data within the reference depth range (520) without interference from the external object (805). Based on identifying that the second depth data (810) of the external object (805) is greater than the reference depth range (520), the at least one processor (210) can perform operation 720 of FIG. 7.

[0075] FIG. 9 is a diagram illustrating an example of changing the display position of a virtual object according to various embodiments.

[0076] Referring to FIG. 9, according to one embodiment, a state (900) may be described as a state before a display position of a virtual object (530) is changed. Within the state (900), at least one processor (210) may display a virtual object (530) having a first size within a virtual 3D space (505) according to first depth data (515). While displaying the virtual object (530) within the virtual 3D space (505), the at least one processor (210) may enter a touch input mode. Based on entering the touch input mode, the at least one processor (210) may identify the first depth data (515) of the virtual object (530) outside the reference depth range (520). At least one processor (210) can identify that the second depth data of the identified external object using one or more cameras is greater than the third depth data (910) within the reference depth range (520) based on identifying that the first depth data (515) is outside the reference depth range (520) (or that the external object is not located according to the second depth data that is less than the third depth data (910)).

[0077] According to one embodiment, the head-mounted electronic device (200) may transition from state (900) to state (905) based on identifying that the second depth data of the external object is greater than the third depth data (910) (or that the external object is not located according to the second depth data that is less than the third depth data (910). State (905) may be described as a state in which the display position of the virtual object (605) is changed. Within state (905), at least one processor (210) may change the size of the virtual object (510) by adjusting the first size of the virtual object (510) to the second size. Adjusting the first size of the virtual object (510) to the second size may be described and understood with reference to FIG. 6.

[0078] According to one embodiment, at least one processor (210) can change the display position of the virtual object (605) by adjusting the first depth data (515) of the virtual object (510) to the third depth data (910) within the reference depth range (520). The at least one processor (210) can display the virtual object (605) having the second size within the virtual 3D space (505) according to the third depth data (910). Since the virtual object (605) has the second size within the reference size range, the virtual object (605) can be shown to the user at a size at which the user can perform a touch input. Since the virtual object (605) is displayed according to the third depth data (910) within the reference depth range (520), the user (501) can perform a touch input to the virtual object (605) without moving in a direction toward the virtual object (605) (or without bending an arm).

[0079] According to one embodiment, at least one processor (210) can display a virtual object (605) at a height corresponding to a height at which the head-worn electronic device (200) is positioned within a virtual 3D space. By displaying the virtual object (605) at a height corresponding to the height at which the head-worn electronic device (200) is positioned, the user (501) can perform a touch input to the virtual object (605) by extending an arm in a straight direction.

[0080] According to one embodiment, while another virtual object is displayed according to depth data less than the third depth data (910) within the virtual 3D space (505), the display position of the virtual object (605) may be changed. As the virtual object (605) is displayed according to the third depth data (910) within the virtual 3D space (505), at least a portion of the virtual object (605) may not be visible to the user (501) by the other virtual object being displayed according to depth data less than the third depth data (910). At least one processor (210) may have an error in receiving a touch input for the virtual object (605) displayed according to the third depth data (910) by the other virtual object being displayed according to depth data less than the third depth data (910). To resolve such errors, at least one processor (210) may perform blur processing on other virtual objects displayed according to depth data smaller than the third depth data (910) based on displaying a virtual object (605) according to the third depth data (910) within the virtual 3D space (505), and may stop (or refrain from, or not receive) receiving touch input for the other virtual objects.

[0081] FIG. 10 is a flowchart illustrating exemplary operations of a head-worn electronic device for comparing second depth data of an external object with reference depth data according to various embodiments.

[0082] Referring to FIG. 10, according to one embodiment, in operation 1000, at least one processor (210) may compare the second depth data of the external object with the reference depth data based on identifying that the second depth data of the external object is smaller than the reference depth range. Operation 1000 may correspond to operation 730 of FIG. 7. According to one embodiment, when the external object is positioned relatively close to the user, there may be an error in receiving a touch input for a virtual object displayed in front of the external object. To resolve this error, it may be required to display a virtual object next to the external object.

[0083] According to one embodiment, in operation 1010, at least one processor (210) can identify whether the second depth data of the external object is greater than the reference depth data by comparing the second depth data of the external object with the reference depth data.

[0084] According to one embodiment, in operation 1020, at least one processor (210) can adjust the first depth data of the virtual object to fourth depth data that is smaller than the second depth data of the external object and larger than the reference depth data based on second depth data of the external object that is larger than the reference depth data. The at least one processor (210) can change the display position of the virtual object by adjusting the first depth data of the virtual object to the fourth depth data. The at least one processor (210) can display the virtual object in front of the external object by displaying the virtual object in the virtual 3D space according to the fourth depth data that is smaller than the second depth data of the external object.

[0085] According to one embodiment, at least one processor (210) can receive a touch input for a virtual object without interference from an external object by displaying the virtual object in a virtual 3D space according to fourth depth data that is smaller than second depth data of an external object. As the fourth depth data is greater than reference depth data defined as minimum depth data at which a user can perform a touch input, the at least one processor (210) can receive a touch input for a virtual object displayed according to the fourth depth data without movement of the user. Displaying a virtual object according to the fourth depth data is illustrated and described in more detail below with reference to FIG. 11.

[0086] According to one embodiment, in operation 1030, at least one processor (210) can move a virtual object next to an external object based on second depth data of the external object that is smaller than the reference depth data. At least one processor (210) can adjust first depth data of the virtual object to the reference depth data based on second depth data of the external object that is smaller than the reference depth data. At least one processor (210) can change a display position of the virtual object by moving the virtual object next to the external object and adjusting the first depth data of the virtual object to the reference depth data. At least one processor (210) can display the virtual object at a position next to the external object based on the reference depth data within a virtual 3D space.

[0087] According to one embodiment, when a virtual object is displayed in a virtual 3D space according to depth data that is smaller than second depth data of an external object that is smaller than the reference depth data, an error may occur in receiving a touch input for the virtual object because the distance between the virtual object and the user is relatively short. When a virtual object is displayed in a virtual 3D space according to the reference depth data, an error may occur in receiving a touch input for the virtual object by an external object positioned according to the second depth data that is smaller than the reference depth data. To resolve these errors, at least one processor (210) may display a virtual object in a position next to an external object in the virtual 3D space according to the reference depth data. Displaying a virtual object in a position next to an external object according to the reference depth data is illustrated and described in more detail below with reference to FIG. 11.

[0088] FIG. 11 is a diagram illustrating an example of changing the display position of a virtual object by comparing second depth data of an external object with reference depth data according to various embodiments.

[0089] Referring to FIG. 11, according to one embodiment, at least one processor (210) may identify whether an external object (805) is positioned based on depth data below a reference depth range based on entering a touch input mode. At least one processor (210) may display a virtual object (e.g., a window, a UI) (605) in front of the external object (805) based on the external object (805) being positioned based on depth data below a reference depth range. FIG. 11 illustrates an example for determining an optimal position at which the virtual object (605) is to be displayed based on the length of the arm of the user (501) and the position of the external object (805) when displaying the virtual object (605). The state (1100) may be described as a state in which the second depth data (810) of the external object (805) is greater than the reference depth data (1105). Within the state (1100), at least one processor (210) can adjust the first depth data (e.g., the first depth data (515) of FIG. 5) of the virtual object (605) to the fourth depth data (1110) based on the second depth data (810) of the external object (805) that is greater than the reference depth data (1105). The fourth depth data (1110) can be less than the second depth data (810) of the external object (805) and greater than the reference depth data (1105). The at least one processor (210) can change the display position of the virtual object (605) by adjusting the first depth data of the virtual object (605) to the fourth depth data (1110). At least one processor (210) can display the virtual object (605) in the virtual 3D space (505) according to the fourth depth data (1110) by changing the display position of the virtual object (605).

[0090] According to one embodiment, the at least one processor (210) displays the virtual object (605) according to fourth depth data (1110) that is smaller than second depth data (810) of the external object (805) within the virtual 3D space (505), so that the virtual object (605) can be positioned in front of the external object (805) within the virtual 3D space (505). As the virtual object (605) is positioned in front of the external object (805) within the virtual 3D space (505), the at least one processor (210) can receive a touch input for the virtual object (605) without interference from the external object (805).

[0091] According to one embodiment, at least one processor (210) displays the virtual object (605) in the virtual 3D space (505) according to fourth depth data (1110) that is greater than the reference depth data (1105), so that the virtual object (605) can be positioned according to depth data that is greater than the minimum depth data at which the user (501) can perform a touch input. Since the fourth depth data (1110) of the virtual object (605) is greater than the reference depth data (1105) that is defined as the minimum depth data at which the user (501) can perform a touch input, receiving a touch input for the virtual object (605) may be error-free.

[0092] According to one embodiment, at least one processor (210) can change the size of the virtual object (605) by adjusting the first size of the virtual object (605) to a second size within a reference size range. By changing the size of the virtual object (605), the at least one processor (210) can display the virtual object (605) having the second size within the virtual 3D space (505) according to the fourth depth data (1110). Since the virtual object (605) displayed according to the fourth depth data (1110) has the second size within the reference size range, the virtual object (605) can be shown to the user as a size at which the user can perform a touch input. Since the virtual object (605) is displayed according to the fourth depth data (1110) that is smaller than the third depth data (e.g., the third depth data (815) of FIG. 8) within the reference depth range (e.g., the reference depth range (520) of FIG. 5), the user (501) can perform a touch input on the virtual object (605) without moving in the direction of the virtual object (605).

[0093] According to one embodiment, the state (1115) can be described as a state in which the second depth data (810) of the external object (805) is smaller than the reference depth data (1105). Within the state (1115), at least one processor (210) can adjust the first depth data of the virtual object (605) to the third depth data (910) based on the second depth data (810) of the external object (805) being smaller than the reference depth data (1105). The at least one processor (210) can change the display position of the virtual object (605) by adjusting the first depth data of the virtual object (605) to the third data (910). The at least one processor (210) can display the virtual object (605) in the virtual 3D space (505) according to the third depth data (910) by changing the display position of the virtual object (605).

[0094] According to one embodiment, when a virtual object (605) is displayed in a frontal direction of a user (501) according to third depth data (910) greater than second depth data (810) of an external object (805) within a virtual 3D space (505), there may be an error in receiving a touch input for the virtual object (605) by an external object (805) positioned in front of the virtual object (605). To resolve this error, at least one processor (210) may display the virtual object (605) within the virtual 3D space (505) according to the third depth data (910) at a position next to the external object (805) rather than in a frontal direction of the user. As the virtual object (605) is positioned next to the external object (805) within the virtual 3D space (505), at least one processor (210) can receive touch input for the virtual object (605) without interference from the external object (805).

[0095] According to one embodiment, at least one processor (210) displays a virtual object (605) in a virtual 3D space (505) according to the third depth data (910), so that the virtual object (605) can be positioned according to optimal depth data at which a user (501) can perform a touch input. Since the virtual object (605) is displayed according to the third depth data (910) that can be referenced as optimal depth data at which a user (501) can perform a touch input, receiving a touch input for the virtual object (605) may be error-free.

[0096] At least one processor (210) can change the size of the virtual object (605) by adjusting the first size of the virtual object (605) to a second size within a reference size range. By changing the size of the virtual object (605), the at least one processor (210) can display the virtual object (605) having the second size within the virtual 3D space (505) next to the external object (805) according to the third depth data (910). Since the virtual object (605) has the second size within the reference size range, the virtual object (605) displayed according to the reference depth data (1105) can be shown to the user as a size at which the user can perform a touch input. Since the virtual object (605) is displayed according to the third depth data (1105) within the reference depth range, the user (501) can perform a touch input on the virtual object (605) without moving in a direction toward the virtual object (605).

[0097] According to one embodiment, at least one processor (210) may refrain from entering (or abort, or bypass, or not enter) the touch input mode based on the second depth data (810) of the external object (805) being smaller than the reference depth data (1105). The at least one processor (210) may display a pop-up window notifying that the touch input mode is not to be entered (or cannot be entered) within the virtual 3D space (505). When the at least one processor (210) displays a virtual object (605) in front of an external object (805) having second depth data (810) smaller than the reference depth data (1105) within the virtual 3D space (505), the display position of the virtual object (605) may be maintained, and the touch input mode may be refrained from entering (or abort, or bypass, or not enter) since a touch input for the virtual object (605) cannot be received.

[0098] According to one embodiment, at least one processor (210) can display a plurality of virtual objects within a virtual 3D space (505). The at least one processor (210) can adjust depth data of the plurality of virtual objects to receive touch inputs for the plurality of virtual objects. Changing display positions of the plurality of virtual objects by adjusting the depth data of the plurality of virtual objects is illustrated and described in more detail below with reference to FIG. 12.

[0099] FIG. 12 is a diagram illustrating an example of changing the display positions of multiple virtual objects according to various embodiments.

[0100] Referring to FIG. 12, according to one embodiment, a state (1200) may be described as a state before display positions of a plurality of virtual objects (e.g., a virtual object (510) and another virtual object (1205)) are changed. Within the state (1200), at least one processor (210) may display the virtual object (510) and another virtual object (1205) within a virtual 3D space (505). While the virtual object (510) and another virtual object (1205) are displayed within the virtual 3D space (505), at least one processor (210) may enter a touch input mode.

[0101] According to one embodiment, at least one processor (210) can identify first depth data (515) of a virtual object (510) and fifth depth data (1210) of another virtual object (1205) based on entering a touch input mode. At least one processor (210) can identify that the first depth data (515) and the fifth depth data (1210) are outside a reference depth range (520). Based on identifying that the first depth data (515) and the fifth depth data (1210) are outside the reference depth range (520), the head-mounted electronic device (200) can transition from state (1200) to state (1215).

[0102] According to one embodiment, the state (1215) can be described as a state in which the display positions of a plurality of virtual objects (e.g., the virtual object (605) and the virtual object (1220)) are changed. Within the state (1215), at least one processor (210) can adjust the first depth data (515) of the virtual object (510) to the third depth data (910) within the reference depth range (520), and adjust the fifth depth data (1210) of the other virtual object (1205) to the sixth depth data (1225) within the reference depth range (520), based on identifying that the first depth data (515) and the fifth depth data (1210) are outside the reference depth range (520). At least one processor (210) can change the display position of the virtual object (510) by adjusting the first depth data (515) of the virtual object (510) to the third depth data (910). At least one processor (210) can change the display position of another virtual object (1205) by adjusting the fifth depth data (1210) of another virtual object (1205) to the sixth depth data (1225). At least one processor (210) can change the display positions of the virtual object (510) and another virtual object (1205) so that the virtual object (510) and another virtual object (1205) can be displayed in a line in the front direction of the user. By displaying the virtual object (510) and another virtual object (1205) in a line in the front direction of the user, the user's field of vision can be relatively less obstructed, or a relatively wide space within the virtual 3D space (505) can be shown to the user.

[0103] According to one embodiment, at least one processor (210) can adjust a first size of a virtual object (510) to a second size within a reference size range. At least one processor (210) can adjust a third size of another virtual object (1205) to a fourth size within a reference size range. An aspect ratio of another virtual object (1205) having the third size can correspond to an aspect ratio of another virtual object (1220) having the fourth size.

[0104] According to one embodiment, at least one processor (210) can display a virtual object (605) having a second size according to third depth data (910) and display another virtual object (1220) having a fourth size according to sixth depth data (1225) within a virtual 3D space (505). Since the virtual object (605) displayed according to the third depth data (910) has the second size within a reference size range, the virtual object (605) can be shown to the user as a size at which the user can perform a touch input. Since the other virtual object (1220) displayed according to the sixth depth data (1225) has the fourth size within the reference size range, the other virtual object (1220) can be shown to the user as a size at which the user can perform a touch input. At least a portion of the other virtual object (1220) can be shown to the user (501) by not overlapping the virtual object (605). At least one processor (210) can recognize the hand of the user (501) in contact with at least a portion of another virtual object (1220) shown to the user (501) as a touch input to the other virtual object (1220).

[0105] According to one embodiment, since the virtual object (605) is displayed according to the third depth data (910) within the reference depth range (520), the user (501) can perform a touch input to the virtual object (605) without moving in the direction of the virtual object (605) (or without bending the arm). Since another virtual object (1220) is displayed according to the sixth depth data (1225) within the reference depth range (520), the user (501) can perform a touch input to the other virtual object (1220) without moving in the direction of the other virtual object (1220) (or without bending the arm). At least one processor (210) can adjust the sixth depth data (1225) of the other virtual object (1220) to the third depth data (910) and adjust the third depth data (910) of the virtual object (605) to the sixth depth data (1225) based on a touch input to the other virtual object (1220). At least one processor (210) can adjust the sixth depth data (1225) of the virtual object (1220) to the third depth data (910) and adjust the third depth data (910) of the virtual object (605) to the sixth depth data (1225), thereby changing the display position of the other virtual object (1220) to the display position of the virtual object (605) and changing the display position of the virtual object (605) to the display position of the other virtual object (1220). At least one processor (210) can change the display position of another virtual object (1220) to the display position of the virtual object (605), and change the display position of the virtual object (605) to the display position of another virtual object (1220), thereby displaying another virtual object (1220) in front of the virtual object (605) within the virtual 3D space (505).At least one processor (210) can provide a function mapped to another virtual object (1220) based on receiving a touch input to the other virtual object (1220) by displaying the other virtual object (1220) in front of the virtual object (605) within the virtual 3D space (505).

[0106] According to one embodiment, at least one processor (210) can display an executable object next to a virtual object (605) within the virtual 3D space (505). Based on receiving a touch input for the executable object, the at least one processor (210) can change the display position of the virtual object (605) to a display position of another virtual object (1220), and change the display position of the other virtual object (1220) to a display position of the virtual object (605). The at least one processor (210) can display another virtual object (1220) in front of the virtual object (605) within the virtual 3D space (505), and provide a function mapped to the other virtual object (1220) based on receiving a touch input for the other virtual object (1220).

[0107] According to one embodiment, at least one processor (210) may receive an input for selecting one virtual object from among a plurality of virtual objects (605, 1220). Based on the input for selecting one virtual object from among a plurality of virtual objects (605, 1220), at least one processor (210) may display the selected virtual object in front of the user within a reference depth range and display the remaining virtual objects excluding the selected virtual object behind the selected virtual object. According to one embodiment, although the display position of the virtual object (605) and the display position of the other virtual object (1220) are changed according to the fact that no external object is positioned in the front direction of the user (501) in FIG. 12, the display position of the virtual object (605) and the display position of the other virtual object (1220) may be changed according to the second depth data of the external object illustrated and described with reference to FIG. 11.

[0108] According to one embodiment, the third depth data (910) of the virtual object (605) may change according to the movement of the user (501) within the virtual 3D space (505). It may be required to adjust the depth data of the virtual object (605) that changes according to the movement of the user (501). Depending on the change in the direction of the head of the user (501), the virtual object (605) may not be positioned in the frontal direction of the user (501) within the virtual 3D space (505). It may be required to maintain the display position of the virtual object (605) that changes according to the direction of the head of the user (501). Maintaining the display position of the virtual object (605) according to the movement of the user and the change in the direction of the user's head is illustrated and described in more detail below with reference to FIG. 13.

[0109] FIG. 13 illustrates an example of maintaining the display position of a virtual object according to changes in the movement of the user and the direction of the user's head according to various embodiments.

[0110] Referring to FIG. 13, according to one embodiment, a state (1300) may be described as a state in which a virtual object (605) is displayed within a virtual 3D space (505) according to third depth data (910) within a reference depth range (520) based on entering a touch input mode. Within the state (1300), at least one processor (210) may identify a movement of the user (501) and / or a change in the direction (1305) of the head of the user (501) while displaying the virtual object (605) within the virtual 3D space (505) according to the third depth data (910). As the user (501) moves while the virtual object (605) is displayed within the virtual 3D space (505), the third depth data (910) of the virtual object (605) may change. By changing the third depth data (910) of the virtual object (605), the user (501) can perform a touch input on the virtual object (605) by moving in the direction of the virtual object (605) or bending his / her arm. In order to resolve the inconvenience of the user (501) due to the change in the third depth data (910) of the virtual object (605), at least one processor (210) can adjust the depth data of the virtual object (605) that changes according to the movement of the user (501) to the third depth data (910). By adjusting the depth data of the virtual object (605) to the third depth data (910), the depth data of the virtual object (605) can be maintained as the third depth data (910) even if the user (501) moves. At least one processor (210) can maintain the display position of the virtual object (605) within the virtual 3D space (505) by maintaining the depth data of the virtual object (605) as third depth data (910).At least one processor (210) maintains the display position of the virtual object (605) within the virtual 3D space (505), so that the user (501) can perform a touch input to the virtual object (605) without moving in the direction of the virtual object (605) (or without bending the arm).

[0111] According to one embodiment, while a virtual object (605) is displayed in a virtual 3D space (505), as the direction (1305) of the head of the user (501) changes, the virtual object (605) may be displayed in a direction other than the front direction of the user (501). By displaying the virtual object (605) in a different direction from the user (501), the user (501) may perform a touch input to the virtual object (605) by changing the gaze in the direction of the virtual object (605) or rotating the body (or head). In order to resolve the inconvenience of the user (501) due to the change in the display position of the virtual object (605), at least one processor (210) may adjust the display position of the virtual object (605) in the virtual 3D space (505) that changes according to the change in the head direction (1305) of the user (501) to the front direction of the user (501). At least one processor (210) can adjust the display position of the virtual object (605) to the front direction of the user (501), so that even if the user (501) changes the direction of the head (1305), the display position of the virtual object (605) can be maintained in the front direction of the user. At least one processor (210) can maintain the display position of the virtual object (605) within the virtual 3D space (505), so that the user (501) can perform a touch input to the virtual object (605) without changing the gaze in the direction of the virtual object (605) or rotating the body (or head).

[0112] According to one embodiment, although the display position of the virtual object (605) is changed according to the external object not being positioned in the front direction of the user (501) within FIG. 12, the display position of the virtual object (605) may be changed according to the second depth data of the external object, which is illustrated and described with reference to FIG. 11. At least one processor (210) may display the virtual object (605) according to the fourth depth data that is smaller than the second depth data of the external object, when the external object is positioned in the front direction of the user (501) according to the second depth data that is smaller than the third depth data (910) according to the movement of the user (501) and / or the change in the direction of the head of the user (501), while the virtual object (605) is displayed according to the third depth data (910) within the reference depth range (520) within the virtual 3D space (505). At least one processor (210) may display a virtual object (605) according to fourth depth data that is smaller than second depth data of an external object within a virtual 3D space (505), and when the external object is not positioned in the frontal direction of the user (501) due to movement of the user (501) and / or change in direction of the head of the user (501), the virtual object (605) may be displayed according to third depth data (910) within a reference depth range (520).

[0113] According to one embodiment, at least one processor (210) may exit the touch input mode while displaying a virtual object (605) within a virtual 3D space (505) according to third depth data (910) within a reference depth range (520) based on entering the touch input mode. The display position of the virtual object that changes based on exiting the touch input mode is illustrated and described with reference to FIG. 14.

[0114] FIG. 14 is a flowchart illustrating exemplary operations of a head-worn electronic device for re-changing the display position of a virtual object according to various embodiments.

[0115] Referring to FIG. 14, according to one embodiment, in operation 1400, at least one processor (210) may terminate the touch input mode while the display position of the virtual object has changed based on entering the touch input mode. The at least one processor (210) may terminate the touch input mode based on a user input. As a non-limiting example, the user input for terminating the touch input mode may include an input to a virtual object (or a virtual button) within a virtual 3D space. The at least one processor (210) may terminate the touch input mode by switching from the touch input mode to another input mode. The at least one processor (210) may receive an input to the virtual object based on a user gesture (e.g., a pinch gesture) performed while the user's hand is away from the virtual object within the other input mode.

[0116] According to one embodiment, at least one processor (210) may terminate the touch input mode for some virtual objects among a plurality of virtual objects displayed in a virtual 3D space. By terminating the touch input mode for some of the virtual objects, at least one processor (210) may receive an input for some of the virtual objects based on a user gesture, and may receive a touch input for the remaining virtual objects among the plurality of virtual objects.

[0117] According to one embodiment, in operation 1410, at least one processor (210) may adjust the second size of the virtual object to the first size based on terminating the touch input mode. The at least one processor (210) may change the size of the virtual object again by adjusting the second size of the virtual object to the first size. In order to change the size of the virtual object again, the at least one processor (210) may store the first size of the virtual object before the size change in the memory (220) based on entering the touch input mode.

[0118] According to one embodiment, in operation 1420, at least one processor (210) may change the display position of the virtual object again by adjusting the third depth data of the virtual object to the first depth data. In order to change the display position of the virtual object again, the at least one processor (210) may store the first depth data of the virtual object before the display position is changed in the memory (220) based on entering the touch input mode. Changing the size and display position of the virtual object again is illustrated and described in more detail below with reference to FIG. 15.

[0119] FIG. 15 is a diagram illustrating an example of changing the size and display position of a virtual object according to various embodiments.

[0120] Referring to FIG. 15, according to one embodiment, a state (1500) may be described as a state before the touch input mode is terminated. Within the state (1500), at least one processor (210) may, while entering the touch input mode, display a virtual object (605) having a second size within a virtual 3D space (505) according to third depth data (910) within a reference depth range (520). Based on entering the touch input mode, at least one processor (210) may store, within the memory (220), the first size of the virtual object (605) before being changed to the second size and the first depth data (e.g., the first depth data (515) of FIG. 5) before being changed to the third depth data (910) of the virtual object (605). At least one processor (210) may terminate the touch input mode while displaying a virtual object (605) having a second size according to third depth data (910) within a virtual 3D space (505). Based on terminating the touch input mode, the head-mounted electronic device (200) may transition from state (1500) to state (1505).

[0121] According to one embodiment, the state (1505) may be described as a state in which the touch input mode is terminated. Within the state (1505), at least one processor (210) may change the size of the virtual object (510) by adjusting the second size of the virtual object (510) to the first size based on terminating the touch input mode. At least one processor (210) may change the display position of the virtual object (510) by adjusting the third depth data (910) of the virtual object (510) to the first depth data (515) based on terminating the touch input mode. At least one processor (210) may retrieve the first size and first depth data (515) stored in the memory (220) based on terminating the touch input mode.

[0122] According to one embodiment, at least one processor (210) can display a virtual object (510) having a first size within a virtual 3D space (505) according to first depth data (515). Even if the virtual object (510) is displayed according to the first depth data (515), at least one processor (210) can receive an input for the virtual object (510) based on a user gesture (e.g., a pinch gesture) performed while the hand of the user (501) is away from the virtual object (510) within another input mode.

[0123] FIG. 16 is a block diagram illustrating examples of configurations of head-worn electronic devices according to various embodiments.

[0124] Referring to FIG. 16, the head-mounted electronic device (200) may include a mode management unit (1600) (e.g., including various circuits and / or executable program instructions), a pose management unit (1610) (e.g., including various circuits and / or executable program instructions), and / or a locator unit (1620) (e.g., including various circuits and / or executable program instructions). The mode management unit (1600), the pose management unit (1610), and / or the locator unit (1620) may support a function of processing a virtual object through an algorithm stored in the memory (220). Although the mode management unit (1600), the pose management unit (1610), and / or the locator unit (1620) are described as “units,” they may perform the following functions in software and / or functionally.

[0125] According to one embodiment, the mode management unit (1600) may perform a function of managing the modes of applications running in the head-mounted electronic device (200). The mode management unit (1600) may display a screen for setting the mode through the display assembly (240). The mode management unit (1600) may enter a touch input mode while a virtual object provided from an application is displayed in a virtual 3D space. Based on entering the touch input mode, the mode management unit (1600) may store the depth data and the size of the virtual object before the display position is changed in the memory (220). Based on terminating the touch input mode, the mode management unit (1600) may retrieve the depth data and the size of the virtual object stored in the memory (220).

[0126] According to one embodiment, the posture management unit (1610) may identify depth data of a virtual object to change the display position of the virtual object. The posture management unit (1610) may identify whether the depth data of the virtual object is within a reference depth range to change the display position of the virtual object. The posture management unit (1610) may identify depth data of an external object to display the virtual object in front of the external object.

[0127] According to one embodiment, the locator unit (1620) can change the display position of a virtual object based on entering the touch input mode. The locator unit (1620) can change the size of the virtual object based on entering the touch input mode.

[0128] FIG. 17 is a block diagram of an electronic device within a network environment according to various embodiments.

[0129] Referring to FIG. 17, in a network environment (1700), an electronic device (1701) may communicate with an electronic device (1702) via a first network (1798) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (1704) or a server (1708) via a second network (1799) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (1701) may communicate with the electronic device (1704) via the server (1708). According to one embodiment, the electronic device (1701) may include a processor (1720), a memory (1730), an input module (1750), an audio output module (1755), a display module (1760), an audio module (1770), a sensor module (1776), an interface (1777), a connection terminal (1778), a haptic module (1779), a camera module (1780), a power management module (1788), a battery (1789), a communication module (1790), a subscriber identification module (1796), or an antenna module (1797). In some embodiments, the electronic device (1701) may omit at least one of these components (e.g., the connection terminal (1778)), or may have one or more other components added. In some embodiments, some of these components (e.g., sensor module (1776), camera module (1780), or antenna module (1797)) may be integrated into a single component (e.g., display module (1760)).

[0130] The processor (1720) may execute software (e.g., a program (1740)) to control at least one other component (e.g., a hardware or software component) of the electronic device (1701) connected to the processor (1720) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (1720) may store commands or data received from other components (e.g., a sensor module (1776) or a communication module (1790)) in a volatile memory (1732), process the commands or data stored in the volatile memory (1732), and store result data in a non-volatile memory (1734). According to one embodiment, the processor (1720) may include a main processor (1721) (e.g., a central processing unit or an application processor) or a secondary processor (1723) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (1721). For example, when the electronic device (1701) includes the main processor (1721) and the secondary processor (1723), the secondary processor (1723) may be configured to use less power than the main processor (1721) or to be specialized for a given function. The secondary processor (1723) may be implemented separately from the main processor (1721) or as a part thereof. Accordingly, the processor (1720) may include various processing circuits and / or multiple processors. For example, as used herein, the term "processor" including the claims includes various processing circuits, including at least one processor, and one or more of the at least one processors may be configured to individually and / or collectively perform the various functions described herein in a distributed manner.As used herein, when "a processor," "at least one processor," and "one or more processors" are described as being configured to perform a number of functions, these terms encompass, for example and without limitation, situations where one processor performs some of the recited functions and other processor(s) perform other of the recited functions, and also situations where a single processor can perform all of the recited functions. Additionally, the at least one processor may comprise a combination of processors that perform the various recited / disclosed functions (e.g., in a distributed manner). At least one processor may execute program instructions to accomplish or perform the various functions.

[0131] The auxiliary processor (1723) may control at least a portion of functions or states associated with at least one component (e.g., a display module (1760), a sensor module (1776), or a communication module (1790)) of the electronic device (1701) on behalf of the main processor (1721) while the main processor (1721) is in an inactive (e.g., sleep) state, or together with the main processor (1721) while the main processor (1721) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (1723) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (1780) or a communication module (1790)). In one embodiment, the auxiliary processor (1723) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (1701) where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (1708)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0132] The memory (1730) can store various data used by at least one component (e.g., the processor (1720) or the sensor module (1776)) of the electronic device (1701). The data can include, for example, software (e.g., the program (1740)) and input data or output data for commands related thereto. The memory (1730) can include volatile memory (1732) or non-volatile memory (1734).

[0133] The program (1740) may be stored as software in memory (1730) and may include an operating system (1742), middleware (1744), or an application (1746).

[0134] The input module (1750) can receive commands or data to be used in a component of the electronic device (1701) (e.g., a processor (1720)) from an external source (e.g., a user) of the electronic device (1701). The input module (1750) can include a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0135] The audio output module (1755) can output audio signals to the outside of the electronic device (1701). The audio output module (1755) can include a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0136] The display module (1760) can visually provide information to an external party (e.g., a user) of the electronic device (1701). The display module (1760) may include a display, a holographic device, or a projector, and a control circuit for controlling the device. In one embodiment, the display module (1760) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0137] The audio module (1770) can convert sound into an electrical signal, or vice versa. According to one embodiment, the audio module (1770) can acquire sound through the input module (1750), output sound through the sound output module (1755), or an external electronic device (e.g., electronic device (1702)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (1701).

[0138] The sensor module (1776) can detect the operating status (e.g., power or temperature) of the electronic device (1701) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (1776) can include a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0139] The interface (1777) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (1701) with an external electronic device (e.g., the electronic device (1702)). In one embodiment, the interface (1777) may include a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0140] The connection terminal (1778) may include a connector through which the electronic device (1701) may be physically connected to an external electronic device (e.g., the electronic device (1702)). In one embodiment, the connection terminal (1778) may include an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0141] The haptic module (1779) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. In one embodiment, the haptic module (1779) can include a motor, a piezoelectric element, or an electrical stimulation device.

[0142] The camera module (1780) can capture still images and videos. In one embodiment, the camera module (1780) may include one or more lenses, image sensors, image signal processors, or flashes.

[0143] The power management module (1788) can manage power supplied to the electronic device (1701). According to one embodiment, the power management module (1788) can be implemented as at least a part of a power management integrated circuit (PMIC).

[0144] A battery (1789) may power at least one component of the electronic device (1701). In one embodiment, the battery (1789) may include a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0145] The communication module (1790) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (1701) and an external electronic device (e.g., electronic device (1702), electronic device (1704), or server (1708)), and the performance of communication through the established communication channel. The communication module (1790) may operate independently from the processor (1720) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (1790) may include a wireless communication module (1792) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (1794) (e.g., a local area network (LAN) communication module, or a power line communication module). Any of these communication modules may communicate with an external electronic device (1704) via a first network (1798) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (1799) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a local area network or a wide area network)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (1792) may use subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (1796) to identify or authenticate the electronic device (1701) within a communication network such as the first network (1798) or the second network (1799).

[0146] The wireless communication module (1792) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency communications (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (1792) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (1792) can support various technologies for securing performance in high-frequency bands, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (1792) can support various requirements specified in the electronic device (1701), an external electronic device (e.g., the electronic device (1704)), or a network system (e.g., the second network (1799)). According to one embodiment, the wireless communication module (1792) can support a peak data rate (e.g., 20 Gbps or more) for eMBB implementation, a loss coverage (e.g., 174 dB or less) for mMTC implementation, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC implementation.

[0147] The antenna module (1797) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (1797) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (1797) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (1798) or the second network (1799), may be selected from the plurality of antennas by the communication module (1790). A signal or power may be transmitted or received between the communication module (1790) and an external electronic device through the selected at least one antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (1797).

[0148] According to various embodiments, the antenna module (1797) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band.

[0149] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0150] According to one embodiment, commands or data may be transmitted or received between the electronic device (1701) and an external electronic device (1704) via a server (1708) connected to a second network (1799). Each of the external electronic devices (1702 or 1704) may be the same or a different type of device as the electronic device (1701). According to one embodiment, all or part of the operations executed in the electronic device (1701) may be executed in one or more of the external electronic devices (1702, 1704, or 1708). When the electronic device (1701) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (1701) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (1701). The electronic device (1701) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (1701) may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (1704) may include an Internet of Things (IoT) device. The server (1708) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (1704) or the server (1708) may be included in a second network (1799).The electronic device (1701) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0151] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0152] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0153] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. In one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0154] Various embodiments of the present document may be implemented as software (e.g., a program (1740)) including one or more instructions stored in a storage medium (e.g., an internal memory (1736) or an external memory (1738)) readable by a machine (e.g., an electronic device (1701)). A processor (e.g., a processor (1720)) of the machine (e.g., an electronic device (1701)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0155] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as included in a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0156] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0157] The technical problems to be achieved in the present disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by a person having ordinary knowledge in the technical field to which the present disclosure pertains.

[0158] As described above, according to one embodiment, a head-worn electronic device (e.g., the head-worn electronic device (200) of FIG. 2) may include at least one processor including a processing circuit (e.g., at least one processor (210) of FIG. 2), a display assembly including a display (e.g., the display assembly (240) of FIG. 2), and a memory (e.g., the memory (220) of FIG. 2) storing one or more programs configured to be individually or collectively executed by the at least one processor, the memory including one or more storage media. The one or more programs may include instructions that cause the head-worn electronic device to display a virtual object (e.g., the virtual object (510) of FIG. 5) within a three-dimensional (3D) space (e.g., the 3D space (505) of FIG. 5) provided through the display assembly. The one or more programs may include instructions that cause the head-worn electronic device to enter a touch input mode that recognizes a user's hand in contact with a user interface (UI) object as a user input while displaying the virtual object in the 3D space. The one or more programs may include instructions that cause the head-worn electronic device to identify first depth data of the virtual object (e.g., first depth data (515) of FIG. 5 ) based on entering the touch input mode. The one or more programs may include instructions that cause the head-worn electronic device to change a display position of the virtual object by adjusting the first depth data of the virtual object to second depth data (e.g., third depth data (910) of FIG. 9 ) within the reference depth range based on identifying that the first depth data of the virtual object is outside a reference depth range (e.g., reference depth range (520) of FIG. 5 ).

[0159] The one or more programs may include instructions that cause the head-mounted electronic device to maintain the display position of the virtual object by maintaining the first depth data of the virtual object based on identifying that the first depth data of the virtual object is within the reference depth range.

[0160] The one or more programs may include instructions that cause the head-worn electronic device to exit the touch input mode while displaying the virtual object in the 3D space according to the second depth data. The one or more programs may include instructions that cause the head-worn electronic device to change the display position of the virtual object again by adjusting the second depth data of the virtual object to the first depth data based on terminating the touch input mode.

[0161] The one or more programs may include instructions that cause the head-worn electronic device to identify a first size of the virtual object based on entering the touch input mode. The one or more programs may include instructions that cause the head-worn electronic device to display the virtual object having the second size in the 3D space according to the second depth data by adjusting the first size of the virtual object to a second size within a reference size range based on identifying that the first depth data of the virtual object is outside the reference depth range.

[0162] The one or more programs may include instructions that cause the head-worn electronic device to identify an aspect ratio of the virtual object based on entering the touch input mode. The one or more programs may include instructions that cause the head-worn electronic device to display the virtual object having the aspect ratio and the second size within the 3D space according to the second depth data by adjusting the first size of the virtual object to the second size while maintaining the aspect ratio based on identifying that the first depth data of the virtual object is outside the reference depth range.

[0163] The one or more programs may include instructions that cause the head-worn electronic device to terminate the touch input mode while displaying the virtual object having the second size in the 3D space according to the second depth data. The one or more programs may include instructions that cause the head-worn electronic device to re-display the virtual object having the first size in the 3D space according to the first depth data by adjusting the second depth data of the virtual object to the first depth data and adjusting the second size of the virtual object to the first size based on the terminating the touch input mode.

[0164] The head-worn electronic device may further include one or more cameras. The one or more programs may include instructions that cause the head-worn electronic device to identify third depth data of an external object using the one or more cameras. The one or more programs may include instructions that cause the head-worn electronic device to compare the third depth data of the external object with the reference depth range based on identifying that the first depth data of the virtual object is outside the reference depth range. The one or more programs may include instructions that cause the head-worn electronic device to change a display position of the virtual object by adjusting the first depth data of the virtual object to fourth depth data that is less than the third depth data and greater than the reference depth data, based on the third depth data of the external object being less than the reference depth range.

[0165] The one or more programs may include instructions that cause the head-mounted electronic device to change a display position of the virtual object by adjusting the first depth data of the virtual object to the second depth data based on the third depth data of the external object that is greater than the reference depth range.

[0166] The one or more programs may include instructions that cause the head-worn electronic device to compare the third depth data of the external object with reference depth data that is less than the second depth data, based on the third depth data of the external object being less than the reference depth range. The one or more programs may include instructions that cause the head-worn electronic device to change a display position of the virtual object so that the virtual object is visible to the user by moving the virtual object next to the external object, and adjusting the first depth data of the virtual object to the second depth data, based on the third depth data of the external object being less than the reference depth data.

[0167] The one or more programs may include instructions that cause the head-worn electronic device to maintain the second depth data of the virtual object by changing the display position of the virtual object according to a change in the position of the user while displaying the virtual object according to the second depth data.

[0168] The one or more programs may include instructions that cause the head-mounted electronic device to identify the direction of the user's head. The one or more programs may include instructions that cause the head-mounted electronic device to change the display position of the virtual object according to the identified direction so that the virtual object is positioned in a frontal direction of the user while displaying the virtual object according to the second depth data.

[0169] The one or more programs may include instructions that cause the head-worn electronic device to enter the touch input mode while displaying the virtual object and another virtual object within the 3D space. The one or more programs may include instructions that cause the head-worn electronic device to identify, based on entering the touch input mode, the first depth data of the virtual object and the third depth data of the another virtual object. The one or more programs may include instructions that cause the head-worn electronic device to change a display position of the virtual object by adjusting the first depth data of the virtual object to the second depth data, and to change a display position of the another virtual object by adjusting the third depth data of the another virtual object to fourth depth data within the reference depth range, based on identifying that the first depth data of the virtual object and the third depth data of the another virtual object are outside the reference depth range.

[0170] The head-worn electronic device may further include one or more cameras. The one or more programs may include instructions that cause the head-worn electronic device to identify, using the one or more cameras, that the hand of the user is in contact with the other virtual object. The one or more programs may include instructions that cause the head-worn electronic device to change a display position of the virtual object by adjusting the second depth data of the virtual object to the fourth depth data based on the identification, and to change a display position of the other virtual object by adjusting the fourth depth data of the other virtual object to the second depth data.

[0171] The head-worn electronic device may further include one or more cameras. The one or more programs may include instructions that cause the head-worn electronic device to identify, using the one or more cameras, that the user's hand is in contact with the virtual object while displaying the virtual object according to the second depth data. The one or more programs may include instructions that cause the head-worn electronic device to provide a function mapped to the virtual object based on the identification.

[0172] The one or more programs may include instructions that cause the head-mounted electronic device to change a display position of the virtual object by adjusting the first depth data of the virtual object to the second depth data based on the first depth data of the virtual object that is outside the identified reference depth range while another virtual object is displayed according to third depth data that is smaller than the second depth data. The one or more programs may include instructions that cause the head-mounted electronic device to perform blur processing on the other virtual object.

[0173] A method as described above, according to one embodiment, may be performed in a head-mounted electronic device including a display assembly. The method may include an operation of displaying a virtual object in a three-dimensional (3D) space provided through the display assembly. The method may include an operation of entering a touch input mode for recognizing a user's hand in contact with a user interface (UI) object as a user input while displaying the virtual object in the 3D space. The method may include an operation of identifying first depth data of the virtual object based on entering the touch input mode. The method may include an operation of changing a display position of the virtual object by adjusting the first depth data of the virtual object to second depth data within the reference depth range based on identifying that the first depth data of the virtual object is outside a reference depth range.

[0174] The method may include an operation of exiting the touch input mode while displaying the virtual object in the 3D space according to the second depth data. The method may include an operation of changing the display position of the virtual object again by adjusting the second depth data of the virtual object to the first depth data based on the termination of the touch input mode.

[0175] The method may include an operation of identifying a first size of the virtual object based on entering the touch input mode. The method may include an operation of adjusting the first size of the virtual object to a second size within a reference size range based on identifying that the first depth data of the virtual object is outside the reference depth range, thereby displaying the virtual object having the second size within the 3D space according to the second depth data.

[0176] The method may include an operation of identifying an aspect ratio of the virtual object based on entering the touch input mode. The method may include an operation of: adjusting the first size of the virtual object to the second size while maintaining the aspect ratio based on identifying that the first depth data of the virtual object is outside the reference depth range; thereby displaying the virtual object having the second size and the aspect ratio within the 3D space according to the second depth data.

[0177] The method may include an operation of terminating the touch input mode while displaying the virtual object having the second size within the 3D space according to the second depth data. The method may include an operation of adjusting the second depth data of the virtual object to the first depth data and adjusting the second size of the virtual object to the first size based on the terminating the touch input mode, thereby displaying the virtual object having the first size within the 3D space again according to the first depth data.

[0178] The head-mounted electronic device may further include one or more cameras. The method may include an operation of identifying third depth data of an external object using the one or more cameras. The method may include an operation of comparing the third depth data of the external object with the reference depth range based on identifying that the first depth data of the virtual object is outside the reference depth range. The method may include an operation of changing a display position of the virtual object by adjusting the first depth data of the virtual object to fourth depth data that is less than the third depth data and greater than the reference depth data based on the third depth data of the external object that is less than the reference depth range.

[0179] The method may include an operation of changing the display position of the virtual object by adjusting the first depth data of the virtual object to the second depth data based on the third depth data of the external object that is greater than the reference depth range.

[0180] The method may include an operation of comparing the third depth data of the external object with reference depth data that is less than the second depth data, based on the third depth data of the external object that is less than the reference depth range. The method may include an operation of moving the virtual object next to the external object, based on the third depth data of the external object that is less than the reference depth data, and changing the display position of the virtual object so that the virtual object is visible to the user by adjusting the first depth data of the virtual object to the second depth data.

[0181] The method may include an operation of maintaining the second depth data of the virtual object by changing the display position of the virtual object according to a change in the position of the user while displaying the virtual object according to the second depth data.

[0182] The method may include an operation of identifying the direction of the user's head. The method may include an operation of changing the display position of the virtual object according to the identified direction so that the virtual object is positioned in the frontal direction of the user while displaying the virtual object according to the second depth data.

[0183] The method may include an operation of entering the touch input mode while displaying the virtual object and another virtual object within the 3D space. The method may include an operation of identifying the first depth data of the virtual object and the third depth data of the other virtual object based on entering the touch input mode. The method may include an operation of changing a display position of the virtual object by adjusting the first depth data of the virtual object to the second depth data based on identifying that the first depth data of the virtual object and the third depth data of the other virtual object are outside the reference depth range, and changing a display position of the other virtual object by adjusting the third depth data of the other virtual object to fourth depth data within the reference depth range.

[0184] The head-mounted electronic device may further include one or more cameras. The method may include an operation of identifying, using the one or more cameras, that the hand of the user is in contact with the other virtual object. The method may include an operation of changing a display position of the virtual object by adjusting the second depth data of the virtual object to the fourth depth data based on the identification, and changing a display position of the other virtual object by adjusting the fourth depth data of the other virtual object to the second depth data.

[0185] The head-mounted electronic device may further include one or more cameras. The method may include an operation of identifying, using the one or more cameras, that the user's hand is in contact with the virtual object while displaying the virtual object based on the second depth data. The method may include an operation of providing a function mapped to the virtual object based on the identification.

[0186] The method may include an operation of changing a display position of the virtual object by adjusting the first depth data of the virtual object to the second depth data based on the first depth data of the virtual object that is outside the identified reference depth range while another virtual object is displayed according to third depth data that is smaller than the second depth data. The method may include an operation of performing blur processing on the other virtual object.

[0187] The non-transitory computer-readable storage medium as described above may store one or more programs. The one or more programs may include instructions that, when executed by a head-worn electronic device including a display assembly, cause the head-worn electronic device to display a virtual object within a three-dimensional (3D) space provided through the display assembly. The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to enter a touch input mode that recognizes a user's hand in contact with a user interface (UI) object as a user input while displaying the virtual object within the 3D space. The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to identify first depth data of the virtual object based on entering the touch input mode. The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to change a display position of the virtual object by adjusting the first depth data of the virtual object to second depth data within the reference depth range based on identifying that the first depth data of the virtual object is outside a reference depth range.

[0188] The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to maintain the display position of the virtual object by maintaining the first depth data of the virtual object based on identifying that the first depth data of the virtual object is within the reference depth range.

[0189] The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to exit the touch input mode while displaying the virtual object in the 3D space according to the second depth data. The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to change the display position of the virtual object again by adjusting the second depth data of the virtual object to the first depth data based on terminating the touch input mode.

[0190] The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to identify a first size of the virtual object based on entering the touch input mode. The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to display the virtual object having the second size in the 3D space according to the second depth data by adjusting the first size of the virtual object to a second size within a reference size range based on identifying that the first depth data of the virtual object is outside the reference depth range.

[0191] The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to identify an aspect ratio of the virtual object based on entering the touch input mode. The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to display the virtual object having the aspect ratio and the second size within the 3D space according to the second depth data by adjusting the first size of the virtual object to the second size while maintaining the aspect ratio based on identifying that the first depth data of the virtual object is outside the reference depth range.

[0192] The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to terminate the touch input mode while displaying the virtual object having the second size in the 3D space according to the second depth data. The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to re-display the virtual object having the first size in the 3D space according to the first depth data by adjusting the second depth data of the virtual object to the first depth data and adjusting the second size of the virtual object to the first size based on terminating the touch input mode.

[0193] The head-worn electronic device may further include one or more cameras. The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to identify third depth data of an external object using the one or more cameras. The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to compare the third depth data of the external object with the reference depth range based on identifying that the first depth data of the virtual object is outside the reference depth range. The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to change a display position of the virtual object by adjusting the first depth data of the virtual object to fourth depth data that is less than the third depth data and greater than the reference depth data, based on the third depth data of the external object being less than the reference depth range.

[0194] The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to change a display position of the virtual object by adjusting the first depth data of the virtual object to the second depth data based on the third depth data of the external object that is greater than the reference depth range.

[0195] The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to compare the third depth data of the external object with reference depth data that is less than the second depth data, based on the third depth data of the external object that is less than the reference depth range. The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to, based on the third depth data of the external object that is less than the reference depth data, move the virtual object next to the external object, and change a display position of the virtual object so that the virtual object is visible to the user by adjusting the first depth data of the virtual object to the second depth data.

[0196] The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to maintain the second depth data of the virtual object by changing the display position of the virtual object according to a change in the position of the user while displaying the virtual object according to the second depth data.

[0197] The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to identify a direction of the user's head. The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to change the display position of the virtual object according to the identified direction while displaying the virtual object according to the second depth data so that the virtual object is positioned in a frontal direction of the user.

[0198] The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to enter the touch input mode while displaying the virtual object and another virtual object in the 3D space. The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to identify the first depth data of the virtual object and the third depth data of the another virtual object based on entering the touch input mode. The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to change a display position of the virtual object by adjusting the first depth data of the virtual object to the second depth data, and to change a display position of the other virtual object by adjusting the third depth data of the other virtual object to fourth depth data within the reference depth range, based on identifying that the first depth data of the virtual object and the third depth data of the other virtual object are outside the reference depth range.

[0199] The head-worn electronic device may further include one or more cameras. The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to identify, using the one or more cameras, that the hand of the user is in contact with the other virtual object. The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to change a display position of the virtual object by adjusting the second depth data of the virtual object to the fourth depth data based on the identification, and to change a display position of the other virtual object by adjusting the fourth depth data of the other virtual object to the second depth data.

[0200] The head-worn electronic device may further include one or more cameras. The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to identify, using the one or more cameras, that the user's hand is in contact with the virtual object while displaying the virtual object according to the second depth data. The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to provide a function mapped to the virtual object based on the identification.

[0201] The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to change a display position of the virtual object by adjusting the first depth data of the virtual object to the second depth data based on the first depth data of the virtual object that is outside the identified reference depth range while another virtual object is displayed according to third depth data that is smaller than the second depth data. The one or more programs may include instructions that, when executed by the head-worn electronic device, cause the head-worn electronic device to perform blur processing on the other virtual object.

[0202] The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects that are not mentioned will be clearly understood by those skilled in the art to which the present disclosure pertains.

[0203] While the present disclosure has been illustrated and described with reference to various exemplary embodiments, it is to be understood that the various exemplary embodiments are intended to be illustrative, not limiting. It will be further understood that various changes in form and detail may be made by those skilled in the art without departing from the true spirit and full scope of the present disclosure, including the appended claims and their equivalents. Furthermore, it will be understood that any of the embodiment(s) disclosed herein may be utilized in conjunction with other embodiment(s) described herein.

Claims

1. In a head-worn electronic device (200), At least one processor (210) comprising a processing circuit; A display assembly (240) including a display; and A memory (220) storing one or more programs configured to be individually or collectively executed by at least one processor (210), and including one or more storage media, One or more of the above programs, Displaying a virtual object (510) within a 3D (three dimensional) space (505) provided through the above display assembly (240), While displaying the virtual object (510) within the above 3D space (505), enter the touch input mode that recognizes the user's hand in contact with the UI (user interface) object as user input, Based on entering the above touch input mode, the first depth data (515) of the virtual object (510) is identified, and Based on identifying that the first depth data (515) of the virtual object (510) is outside the reference depth range (520), the display position of the virtual object (510) is changed by adjusting the first depth data (515) of the virtual object (510) to the second depth data (910) within the reference depth range (520). Including instructions that cause the head-worn electronic device (200) to operate, Head-worn electronic device (200).

2. In claim 1, One or more of the above programs, By maintaining the first depth data (515) of the virtual object (510), based on identifying that the first depth data (515) of the virtual object (510) is within the reference depth range (520), the display position of the virtual object (510) is maintained. Including instructions that cause the head-worn electronic device (200) to operate, Head-worn electronic device (200).

3. In claim 1, One or more of the above programs, While displaying the virtual object (510) in the 3D space (505) according to the second depth data (910), the touch input mode is terminated, and Based on terminating the touch input mode, the display position of the virtual object (510) is changed again by adjusting the second depth data (910) of the virtual object (510) to the first depth data (515). Including instructions that cause the head-worn electronic device (200) to operate, Head-worn electronic device (200).

4. In claim 1, One or more of the above programs, Based on entering the above touch input mode, the first size of the virtual object (510) is identified, and Based on identifying that the first depth data (515) of the virtual object (510) is outside the reference depth range (520), by adjusting the first size of the virtual object (510) to a second size within the reference size range (600), the virtual object (605) having the second size is displayed in the 3D space (505) according to the second depth data (910). Including instructions that cause the head-worn electronic device (200) to operate, Head-worn electronic device (200).

5. In claim 4, One or more of the above programs, Based on entering the above touch input mode, the aspect ratio of the virtual object (510) is identified, and Based on identifying that the first depth data (515) of the virtual object (510) is outside the reference depth range (520), by adjusting the first size of the virtual object (510) to the second size while maintaining the aspect ratio, the virtual object (605) having the second size and the aspect ratio within the 3D space is displayed according to the second depth data (910). Including instructions that cause the head-worn electronic device (200) to operate, Head-worn electronic device (200).

6. In claim 4, One or more of the above programs, While displaying the virtual object (605) having the second size in the 3D space (505) according to the second depth data (910), the touch input mode is terminated, and Based on terminating the touch input mode, the second depth data (910) of the virtual object (605) is adjusted to the first depth data (515), and the second size of the virtual object (510) is adjusted to the first size, thereby displaying the virtual object (510) having the first size in the 3D space (505) again according to the first depth data (515). Including instructions that cause the head-worn electronic device (200) to operate, Head-worn electronic device (200).

7. In claim 1, further comprising one or more cameras (230), One or more of the above programs, Using one or more of the above cameras (230), identify the third depth data (810) of the external object (805), Based on identifying that the first depth data (515) of the virtual object (510) is outside the reference depth range (520), comparing the third depth data (810) of the external object (805) with the reference depth range (520), and By adjusting the first depth data (515) of the virtual object (510) to fourth depth data smaller than the third depth data (810), based on the third depth data (810) of the external object (805) smaller than the reference depth range (520), the display position of the virtual object (510) is changed. Including instructions that cause the head-worn electronic device (200) to operate, Head-worn electronic device (200).

8. In claim 7, One or more of the above programs, By adjusting the first depth data (515) of the virtual object (510) to the second depth data (910), based on the third depth data (810) of the external object (805) that is greater than the reference depth range (520), the display position of the virtual object (510) is changed. Including instructions that cause the head-worn electronic device (200) to operate, Head-worn electronic device (200).

9. In claim 7, One or more of the above programs, Based on the third depth data (810) of the external object (805) that is smaller than the reference depth range (520), the third depth data (810) of the external object (805) is compared with the reference depth data that is smaller than the second depth data (910), and Based on the third depth data (810) of the external object (805) that is smaller than the reference depth data, the virtual object (510) is moved next to the external object (805), and the first depth data (515) of the virtual object (510) is adjusted to the second depth data to change the display position of the virtual object (605) so that the virtual object (605) is visible to the user. Including instructions that cause the head-worn electronic device (200) to operate, Head-worn electronic device (200).

10. In claim 1, One or more of the above programs, While displaying the virtual object (605) according to the second depth data (910), by changing the display position of the virtual object (605) according to a change in the position of the user, the second depth data (910) of the virtual object (605) is maintained. Including instructions that cause the head-worn electronic device (200) to operate, Head-worn electronic device (200).

11. In claim 1, One or more of the above programs, Identify the direction of the user's head, and While displaying the virtual object (605) according to the second depth data (910), change the display position of the virtual object (605) according to the identified direction so that the virtual object (605) is positioned in the front direction of the user. Including instructions that cause the head-worn electronic device (200) to operate, Head-worn electronic device (200).

12. In claim 1, One or more of the above programs, While displaying the virtual object (510) and another virtual object (1205) within the 3D space (505), enter the touch input mode, Based on entering the above touch input mode, identifying the first depth data (515) of the virtual object (510) and the third depth data (1210) of the other virtual object (1205), and Based on identifying that the first depth data (515) of the virtual object (510) and the third depth data (1210) of the other virtual object (1205) are outside the reference depth range (520), the display position of the virtual object (510) is changed by adjusting the first depth data (515) of the virtual object (510) to the second depth data (910), and the display position of the other virtual object (1205) is changed by adjusting the third depth data (1210) of the other virtual object (1205) to the fourth depth data (1225) within the reference depth range (520). Including instructions that cause the head-worn electronic device (200) to operate, Head-worn electronic device (200).

13. In claim 12, further comprising one or more cameras (230), One or more of the above programs, Using one or more of the cameras (230), identifying that the user's hand is in contact with the other virtual object (1220), and Based on the above identification, the display position of the virtual object (605) is changed by adjusting the second depth data (910) of the virtual object (605) to the fourth depth data (1225), and the display position of the other virtual object (1220) is changed by adjusting the fourth depth data (1225) of the other virtual object (1220) to the second depth data (910). Including instructions that cause the head-worn electronic device (200) to operate, Head-worn electronic device (200).

14. A method for executing within a head-worn electronic device (200) comprising a display assembly (240) including a display, the method comprising: An operation of displaying a virtual object (510) within a 3D (three dimensional) space (505) provided through the above display assembly (240), An action of entering a touch input mode that recognizes a user's hand in contact with a UI (user interface) object as a user input while displaying the virtual object (510) within the 3D space (505) An operation of identifying the first depth data (515) of the virtual object (510) based on entering the above touch input mode, and An operation of changing the display position of the virtual object (510) by adjusting the first depth data (515) of the virtual object (510) to second depth data (910) within the reference depth range (520) based on identifying that the first depth data (515) of the virtual object (510) is outside the reference depth range (520). method.

15. In a non-transitory computer-readable storage medium storing one or more programs, The one or more programs, when executed by a head-worn electronic device including a display assembly (240) including a display: Displaying a virtual object (510) within a 3D (three dimensional) space (505) provided through the above display assembly (240), While displaying the virtual object (510) within the above 3D space (505), enter the touch input mode that recognizes the user's hand in contact with the UI (user interface) object as user input, Based on entering the above touch input mode, the first depth data (515) of the virtual object (510) is identified, and Based on identifying that the first depth data (515) of the virtual object (510) is outside the reference depth range (520), the display position of the virtual object (510) is changed by adjusting the first depth data (515) of the virtual object (510) to the second depth data (910) within the reference depth range (520). comprising instructions causing the head-worn electronic device to operate; Non-transitory computer-readable storage medium.

Images