Wearable device and method for displaying user interface, and non-transitory computer-readable storage medium

Abstract

This head-wearable electronic device may comprise: a memory storing instructions; a communication circuit; one or more cameras for obtaining images of the eyes of a user wearing the head-wearable electronic device; a display assembly including at least one display; and at least one processor. The head-wearable electronic device can be instructed to: display, through the display assembly, first UI objects in a manner of superposing same on the environment, in the vicinity of the head-wearable electronic device, viewed through the display assembly; while the first UI objects superimposed on the environment are displayed, use images obtained through the one or more cameras, so as detect a gaze toward another wearable device positioned in the environment viewed through the display assembly and worn by a user of the head-wearable electronic device; and, on the basis of the detection, display, through the display assembly in the environment, a user interface which includes second UI objects and which is arranged to be linked to the other wearable device.

Description

Wearable device, method, and non-transitory computer-readable storage medium for displaying a user interface

[0001] The present disclosure relates to a wearable device, a method, and a non-transitory computer-readable storage medium for displaying a user interface.

[0002] A wearable device may include a camera and a display. The wearable device may be utilized as a tool for implementing virtual reality, augmented reality, and / or mixed reality. For example, the wearable device may be described as AR glasses. For example, the wearable device may be described as a video see-through (VST) device.

[0003] The above information may be provided as background art to aid in understanding the present disclosure.

[0004] No claim or determination is made as to whether any of the above is applicable as prior art to the present disclosure.

[0005] A head-wearable electronic device is described. The head-wearable electronic device may include a memory storing instructions and including one or more storage media. The head-wearable electronic device may include communication circuitry. The head-wearable electronic device may include one or more cameras configured to acquire images of an eye of a user wearing the head-wearable electronic device. The head-wearable electronic device may include a display assembly including at least one display. The head-wearable electronic device may include at least one processor including a processing circuit. The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to display first UI objects through the display assembly, overlaid on an environment surrounding the head-wearable electronic device as seen through the display assembly. The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to detect, using images acquired through the one or more cameras, a gaze toward another wearable device positioned within the environment and worn by a user of the head-wearable electronic device while displaying the first UI objects overlaid on the environment, as viewed through the display assembly. The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to display, on the environment through the display assembly, a user interface comprising second UI objects and arranged in conjunction with the other wearable device, based on the detection.The second UI objects may include UI objects that each provide the functions provided by the first UI objects. The size of each of the second UI objects may be smaller than the size of each of the first UI objects. The user interface may be controlled according to user input received for the other wearable device.

[0006] A method is provided. The method may be implemented within a head-wearable electronic device having a communication circuit, one or more cameras configured to acquire images of an eye of a user wearing the head-wearable electronic device, and a display assembly including at least one display. The method may include displaying, through the display assembly, first UI objects in an overlay on an environment surrounding the head-wearable electronic device as viewed through the display assembly. The method may include detecting, using images acquired through the one or more cameras, a gaze toward another wearable device positioned within the environment viewed through the display assembly and worn by a user of the head-wearable electronic device, while displaying the first UI objects overlaid on the environment. The method may include displaying, through the display assembly, a user interface on the environment, the user interface including second UI objects and arranged in conjunction with the other wearable device, based on the detection. The second UI objects may include UI objects that each provide the functions provided by the first UI objects. The size of each of the second UI objects may be smaller than the size of each of the first UI objects. The user interface may be controlled according to user input received for the other wearable device.

[0007] A non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium may store one or more programs. The one or more programs may include instructions that, when executed by a head-wearable electronic device having a communication circuit, one or more cameras configured to acquire images of an eye of a user wearing the head-wearable electronic device, and a display assembly including at least one display, cause the head-wearable electronic device to display, through the display assembly, first user interface (UI) objects as an overlay on an environment surrounding the head-wearable electronic device as viewed through the display assembly. The one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to detect, using images acquired through the one or more cameras, a gaze toward another wearable device positioned within the environment as viewed through the display assembly and worn by a user of the head-wearable electronic device while displaying the first UI objects as an overlay on the environment. The one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to display, on the environment through the display assembly, a user interface, including second UI objects and arranged in conjunction with the other wearable device, based on the detection. The second UI objects may include UI objects that each provide functions provided by the first UI objects. A size of each of the second UI objects may be smaller than a size of each of the first UI objects.The above user interface can be controlled based on user input received for the other wearable device.

[0008] A head-wearable electronic device is described. The head-wearable electronic device may include a memory storing instructions and including one or more storage media. The head-wearable electronic device may include communication circuitry. The head-wearable electronic device may include one or more first cameras configured to acquire images of an eye of a user wearing the head-wearable electronic device. The head-wearable electronic device may include one or more second cameras configured to acquire images of an environment surrounding the head-wearable electronic device. The head-wearable electronic device may include a display assembly including at least one display. The head-wearable electronic device may include at least one processor including a processing circuit. The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to display, through the display assembly, first UI objects on each of the first images acquired through the one or more second cameras. The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to, while displaying the first UI objects on each of the first images, detect a gaze directed toward a visual object within each of the first images corresponding to another wearable device positioned in an environment surrounding the head-wearable electronic device and worn by a user of the head-wearable electronic device, using second images acquired through the one or more first cameras.The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to display, through the display assembly, a user interface, based on the detection, on each of the first images, the user interface including second UI objects and arranged in conjunction with the visual objects. The second UI objects may include UI objects that each provide functions provided by the first UI objects. A size of each of the second UI objects may be smaller than a size of each of the first UI objects. The user interface may be controlled according to a user input received for the other wearable device.

[0009] A method is provided. The method may be implemented within a head-wearable electronic device having communication circuitry, one or more first cameras configured to acquire images of eyes of a user wearing the head-wearable electronic device, one or more second cameras configured to acquire images of an environment surrounding the head-wearable electronic device, and a display assembly including at least one display. The method may include displaying, via the display assembly, first UI objects on each of the first images acquired through the one or more second cameras. While displaying the first UI objects on each of the first images, the method may include detecting, using the second images acquired through the one or more first cameras, a gaze directed toward a visual object within each of the first images corresponding to another wearable device positioned in the environment surrounding the head-wearable electronic device and worn by the user of the head-wearable electronic device. The method may include, based on the detection, an operation of displaying, through the display assembly, on each of the first images, a user interface, including second UI objects and arranged in conjunction with the visual object. The second UI objects may include UI objects that each provide functions provided by the first UI objects. The size of each of the second UI objects may be smaller than the size of each of the first UI objects. The user interface may be controlled according to a user input received for the other wearable device.

[0010] A non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium may store one or more programs. The one or more programs may include instructions that, when executed by a head-wearable electronic device having a communication circuit, one or more first cameras configured to acquire images of eyes of a user wearing the head-wearable electronic device, one or more second cameras configured to acquire images of an environment surrounding the head-wearable electronic device, and a display assembly including at least one display, cause the head-wearable electronic device to display, through the display assembly, first UI objects on each of the first images acquired through the one or more second cameras. The one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to detect, using second images acquired through the one or more first cameras, a gaze directed toward a visual object in each of the first images that corresponds to another wearable device positioned in an environment surrounding the head-wearable electronic device and worn by a user of the head-wearable electronic device, while displaying the first UI objects on each of the first images. The one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to, based on the detection, display, through the display assembly, a user interface, on each of the first images, that includes second UI objects and is arranged in conjunction with the visual objects. The second UI objects may include UI objects that respectively provide functions provided by the first UI objects.The size of each of the second UI objects may be smaller than the size of each of the first UI objects. The user interface may be controlled based on user input received for the other wearable device.

[0011] Figure 1 illustrates an example of an environment including a head-mounted electronic device.

[0012] Figure 2 is a simplified block diagram of an exemplary head-wearable electronic device.

[0013] FIG. 3 is a flowchart illustrating an exemplary method for displaying a user interface arranged in conjunction with another wearable device.

[0014] Figure 4 illustrates an example of displaying a user interface including second UI objects.

[0015] Figure 5 illustrates an example of controlling a user interface in response to user input.

[0016] FIG. 6 is a flowchart illustrating an exemplary method for activating a mode in which a user interface is controlled based on user input received for another wearable device.

[0017] FIGS. 7A and 7B illustrate examples of a method for activating a mode in which the user interface is controlled based on user input received for the other wearable device.

[0018] FIG. 8 illustrates an example of a method for disabling a mode in which the user interface is controlled based on user input received for the other wearable device.

[0019] FIG. 9 is a flowchart illustrating an exemplary method for authorizing user input received for another wearable device.

[0020] Figure 10 illustrates examples of user input received for different wearable devices.

[0021] Figure 11 illustrates examples of second UI objects moving in response to user input.

[0022] Figure 12 illustrates an example of providing a haptic notification in response to user input.

[0023] Figure 13 illustrates an example of an environment where the head-mounted electronic device is a video see-through (VST) device.

[0024] FIG. 14 is a block diagram of an electronic device within a network environment according to various embodiments.

[0025] Figure 15a shows an example of a perspective view of a wearable device.

[0026] FIG. 15b illustrates an example of one or more hardware elements placed within a wearable device.

[0027] Figures 16a and 16b show an example of the appearance of a wearable device.

[0028] Figure 17 shows an example of a block diagram of a wearable device.

[0029] Fig. 18 shows an example of a block diagram of an electronic device for displaying an image in a virtual space.

[0030] Figure 1 illustrates an example of an environment including a head-mounted electronic device.

[0031] Referring to FIG. 1, an environment (150) may include a head-wearable electronic device (100) worn by a user (120). The head-wearable electronic device (100) may be referred to as a wearable device. For example, the head-wearable electronic device (100) may be used to provide an augmented reality service. For example, the head-wearable electronic device (100) may include at least one processor (e.g., at least one processor (207) of FIG. 2), a display assembly (e.g., a display assembly (208) of FIG. 2), one or more first cameras (e.g., one or more first cameras (209)), and one or more second cameras (e.g., one or more second cameras (210) of FIG. 2)) to provide the augmented reality service.

[0032] For example, if the head-wearable electronic device (100) is AR glasses, the display assembly (e.g., the display assembly (208) of FIG. 2) may include at least one transparent display. For example, at least a portion of an environment (150) located within a user's field of view (FOV) may be viewed through the display assembly (e.g., the display assembly (208) of FIG. 2). For example, the user (120) may wear another wearable device (110). For example, the other wearable device (110) may include a smartwatch. The environment (150) may include the other wearable device (110). For example, the other wearable device (110) may be viewed through the display assembly (e.g., the display assembly (208) of FIG. 2) of the head-wearable electronic device (100). For example, the environment (150) may include a picture frame (160). For example, the environment (150) may include a flower pot (170). For example, the environment (150) may include a drawer (180).

[0033] A head-wearable electronic device (100) can display first user interface (UI) objects (130) through a display assembly (e.g., the display assembly (208) of FIG. 2) overlapping with the environment (150) shown through the display assembly. For example, the head-wearable electronic device (100) can display a user interface (190) including first UI objects (130) through the display assembly (208). For example, the display assembly can include a lens unit and an output unit. For example, the lens unit can include a portion onto which light output from the output unit is projected. For example, the output unit can include a portion that emits light to the lens unit to provide a visual object to a user (120). For example, a UI object can be referred to as an executable object. For example, UI objects can be referred to as executable objects. For example, the first UI objects may be referred to as first executable objects. For example, the second UI objects may be referred to as second executable objects. Each of the first UI objects (130) may correspond to a respective software application. The head-wearable electronic device (100) may execute the software application corresponding to each of the first UI objects (130) based on a user input to the first UI objects (130). For example, each of the first UI objects (130) may be described as a UI object for executing one (a) function. For example, the user input may include a touch input to the head-wearable electronic device (100).While the head-wearable electronic device (100) is worn on the head of the user (120), when the user (120) touches the head-wearable electronic device (100) to perform a touch input, the position of a virtual object (e.g., first UI objects (130)) seen by the user (120) may change because the positional relationship between the head-wearable electronic device (100) and the head is changed. For example, when the user touches the head-wearable electronic device (100), at least a portion of the environment (150) seen through the display assembly (e.g., the display assembly (208) of FIG. 2) may change. For example, in order to prevent or reduce the change in the position of a virtual object seen by the user (120) due to the touch of the user (120), the head-wearable electronic device (100) may be required to receive an input that replaces the touch input. The above input may be input through another wearable device (110) different from the head-mounted electronic device (100).

[0034] A head-wearable electronic device (100) can receive information including a user input from another wearable device (110) through a communication circuit (e.g., the communication circuit (205) of FIG. 2). The head-wearable electronic device (100) can control first UI objects (130) in response to the user input. For example, the head-wearable electronic device (100) can move the position of at least one of the first UI objects (130) in response to the user input. For example, the head-wearable electronic device (100) can execute a software application corresponding to at least one of the first UI objects (130) in response to the user input.

[0035] Another wearable device (110) may be visible through the display assembly (e.g., display assembly (208) of FIG. 2). For example, the other wearable device (110) may be located within an environment (150) visible through the display assembly of the head-mounted electronic device (100) (e.g., display assembly (208) of FIG. 2). While the other wearable device (110) is visible through the display assembly (e.g., display assembly (208) of FIG. 2), the first UI objects (130) may not be visible through the display assembly (e.g., display assembly (208) of FIG. 2). However, the present invention is not limited thereto. While another wearable device (110) is displayed through a display assembly (e.g., display assembly (208) of FIG. 2), the first UI objects (130) may be displayed through the display assembly (e.g., display assembly (208) of FIG. 2). For example, the head-wearable electronic device (100) may stop displaying the first UI objects (130) through the display assembly (e.g., display assembly (208)) and display the second UI objects (e.g., second UI objects (430) of FIG. 4). For example, when the gaze of the user (120) is directed toward the other wearable device (110), the head-wearable electronic device (100) may display the second UI objects (e.g., second UI objects (430) of FIG. 4) in relation to the other wearable device (110).

[0036] For example, the head-mounted electronic device (100) may include hardware components used to perform or execute the above operations. The hardware components are described and exemplified with reference to FIG. 2.

[0037] Figure 2 is a simplified block diagram of an exemplary head-wearable electronic device.

[0038] Referring to FIG. 2, a head-wearable electronic device (100) may include a communication circuit (205), a memory (206), at least one processor (207), a display assembly (208), one or more first cameras (209), and one or more second cameras (210).

[0039] At least one processor (207) may include a hardware component for processing data using instructions stored in the memory (206). The hardware component for processing data may include a central processing unit (CPU) (e.g., including processing circuitry). The hardware component for processing data may include a graphic processing unit (GPU) (e.g., including processing circuitry). The hardware component for processing data may include a display processing unit (DPU) (e.g., including processing circuitry). The hardware component for processing data may include a neural processing unit (NPU) (e.g., including processing circuitry).

[0040] At least one processor (207) may include one or more cores. For example, at least one processor (207) may have a multi-core processor structure such as a dual core, a quad core, or a hexa core.

[0041] The memory (206) may include hardware components for storing data and / or instructions input to and / or output from at least one processor (207). The memory (206) may include, for example, volatile memory such as random-access memory (RAM), and / or non-volatile memory such as read-only memory (ROM). The volatile memory may include, for example, at least one of dynamic RAM (DRAM), static RAM (SRAM), cache RAM, and pseudo SRAM (PSRAM). The non-volatile memory may include, for example, at least one of programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, hard disk, compact disc, and embedded multimedia card (EMMC).

[0042] The communication circuit (205) may include hardware components for supporting transmission and / or reception of signals between the head-mounted electronic device (100) and an external electronic device. The communication circuit (205) may include, for example, at least one of a modem, an antenna, and an optical / electronic (O / E) converter. The communication circuit (205) may support transmission and / or reception of signals based on various types of protocols, such as Ethernet, a local area network (LAN), a wide area network (WAN), wireless fidelity (WiFi), Bluetooth, Bluetooth low energy (BLE), zigbee, long term evolution (LTE), and 5G new radio (NR).

[0043] The display assembly (208) can output visualized information. For example, the display assembly (208) can output visualized information to the user under the control of at least one processor (207). The display assembly (208) can include hardware components of a head-mounted electronic device (100) used to display a screen. For example, the display assembly (208) can include light-emitting elements and circuits (e.g., transistors) that control the light-emitting elements to emit light. For example, each of the light-emitting elements can include an organic light emitting diode (OLED) or a micro LED. However, the present invention is not limited thereto. For example, the display assembly (208) can include a liquid crystal display (LCD).

[0044] As a non-limiting example, the display assembly (208) may include a first display positioned in front of the left eye of a user wearing the head-wearable electronic device (100) and a second display positioned in front of the right eye of the user wearing the head-wearable electronic device (100). For example, first content provided through a screen displayed through the first display may be (substantially) identical to second content provided through a screen displayed through the second display. While the first content and the second content are identical, the screen displayed through the second display may have a disparity with respect to the screen displayed through the first display. For example, the disparity may cause the display assembly (208) to present content (e.g., corresponding to the first content and the second content) in three dimensions.

[0045] The one or more first cameras (209) may include one or more optical sensors (e.g., a charged coupled device (CCD) sensor, a complementary metal oxide semiconductor (CMOS) sensor) that generate an electrical signal representing the color and / or brightness of light. For example, the one or more first cameras (209) may be described as one or more image sensors. For example, at least a portion of the one or more first cameras (209) may have a field of view (FOV) that corresponds to an FOV of a user's eye. For example, the FOV of a portion of the one or more first cameras (209) may be different from the FOV of another portion of the one or more first cameras (209). For example, the one or more first cameras (209) may be used to acquire images of the eyes of a user (120) wearing the head-wearable electronic device (100). For example, one or more first cameras (209) may be arranged toward the eyes of the user (120). For example, one or more first cameras (209) may be included in a camera module (e.g., camera module (1480) of FIG. 14).

[0046] The one or more second cameras (210) may include one or more optical sensors (e.g., a charged coupled device (CCD) sensor, a complementary metal oxide semiconductor (CMOS) sensor) that generate an electrical signal representing the color and / or brightness of light. For example, the one or more second cameras (210) may be described as one or more image sensors. For example, each of the one or more second cameras (210) may be available to acquire an image of the environment surrounding the head-wearable electronic device (100). For example, at least a portion of the one or more first cameras (209) may have a field of view (FOV) corresponding to a field of view (FOV) of a user's eye. For example, the FOV of a portion of the one or more second cameras (210) may be different from the FOV of another portion of the one or more second cameras (210). For example, one or more second cameras (210) may be used to identify input means positioned around the head-mounted electronic device (100). For example, one or more second cameras (210) may be included in a camera module (e.g., camera module (1480) of FIG. 14).

[0047] The environment (150) can be viewed through the display assembly (208). First UI objects (130) can be displayed through the display assembly (208). One or more first cameras (209) can acquire images. At least one processor (207) can detect a gaze of a user (120) using the images. The gaze can be directed toward another wearable device (110) located within the environment (150) viewed through the display assembly (208). Based on the gaze, the at least one processor (207) can display second UI objects (e.g., second UI objects (430) of FIG. 4) through the display assembly (208). At least one processor (207) can display a user interface (e.g., a user interface (490) of FIG. 4) including second UI objects (e.g., second UI objects (430) of FIG. 4) based on the gaze through the display assembly (208). The second UI objects can be arranged in relation to another wearable device (110). The gaze can be directed toward a visual affordance (e.g., a visual affordance (730) of FIG. 7A) displayed through the display assembly (208). One or more second cameras (210) can acquire images. At least one processor (207) can identify a gesture using the images. For example, one or more second cameras (210) can be included in a camera (e.g., a camera (1560) of FIG. 15B). For example, one or more second cameras (210) may be used to identify a gesture associated with another wearable device (110) using images acquired through the one or more second cameras (210). At least one processor may control a user interface (e.g., the user interface (490) of FIG. 4) based on the identification.

[0048] FIG. 3 is a flowchart illustrating an exemplary method for displaying a user interface arranged in conjunction with another wearable device. This method may be executed by the head-mounted electronic device (100) illustrated in FIG. 2 or by at least one processor (207) of the head-mounted electronic device (100).

[0049] Referring to FIG. 3, in operation 310, the head-wearable electronic device (100) may display first UI objects (130) through the display assembly (208) as an overlay on an environment (150) surrounding the head-wearable electronic device (100) shown through the display assembly (208). For example, the UI object may be referred to as an executable object. For example, the UI objects may be referred to as executable objects. For example, the first UI objects may be referred to as first executable objects. For example, the second UI objects may be referred to as second executable objects. For example, the first UI object (e.g., the first UI object (130-1) of FIG. 4) may be displayed through the display assembly (208). For example, the first UI object (e.g., the first UI object (130-2) of FIG. 4) may be displayed through the display assembly (208). For example, a first UI object (e.g., a first UI object (130-3) of FIG. 4) may be displayed through the display assembly (208). For example, a first UI object (e.g., a first UI object (130-4) of FIG. 4) may be displayed through the display assembly (208). For example, a first UI object (e.g., a first UI object (130-5) of FIG. 4) may be displayed through the display assembly (208). For example, a first UI object (e.g., a first UI object (130-6) of FIG. 4) may be displayed through the display assembly (208). Each of the first UI objects (130) may correspond to a respective software application. For example, the software application may include a software application for making calls. For example, the software application may include a software application for email. For example, the software application may include a software application for taking pictures. However, the present invention is not limited thereto.The display of the above first UI objects (130) is described and illustrated in more detail with reference to FIG. 4.

[0050] Figure 4 illustrates an example of displaying a user interface including second UI objects.

[0051] Referring to FIG. 4, a state (410) may be described as a state in which first UI objects (130) are displayed through a display assembly (208). An environment (150) may be displayed through a display assembly (208) of a head-wearable electronic device (100). The head-wearable electronic device (100) may display first UI objects (130) as an overlay on the environment (150) displayed through the display assembly (208). A user interface (190) may include first UI objects (130). The first UI objects (130) may include a first UI object (130-1), a first UI object (130-2), a first UI object (130-3), a first UI object (130-4), a first UI object (130-5), and a first UI object (130-6). However, the present invention is not limited thereto.

[0052] A head-wearable electronic device (100) can modify a user interface (190) including first UI objects (130). For example, the head-wearable electronic device (100) can receive information including user input from another wearable device (110) via the communication circuit (205). For example, the head-wearable electronic device (100) can modify the user interface (190) based on a user input included in the information received from the other wearable device (110). For example, the user input can include a user input received through an input device of the other wearable device (110) (e.g., the input device (1030) of FIG. 10). For example, the input device can be rotatable. For example, the input device can include a bezel. For example, the input device can include a crown of the other wearable device (110). For example, the head-mounted electronic device (100) may display a modified user interface (e.g., the user interface (490) of FIG. 4) including first UI objects (130) through the display assembly (208) in response to the user input. For example, the shape of the modified user interface may be circular.

[0053] For example, the head-wearable electronic device (100) may include two or more modes. For example, the modes may include a first mode and a second mode. The first mode may be described as a mode that controls a user interface (e.g., the user interface (490) of FIG. 4) displayed through the display assembly (208) according to a user input received by the head-wearable electronic device (100). For example, the second mode may be described as a mode that controls the user interface according to a user input included in information received from another wearable device (110). For example, the first mode may be referred to as a normal mode or a stand-alone mode. For example, the second mode may be referred to as a slave mode. The head-wearable electronic device (100) may receive information including a user input from another wearable device (110) through the communication circuit (205).

[0054] For example, another wearable device (110) may include two or more modes. For example, the modes may include a third mode and a fourth mode. The third mode may be described as a mode that controls a user interface displayed on a display of another wearable device based on user input received by the other wearable device (110). For example, the fourth mode may be described as a mode that transmits information including user input received by the other wearable device (110) to the head-mounted electronic device (100). For example, the fourth mode may be referred to as an input mode or a master mode.

[0055] Referring again to FIG. 3 , at operation 320, while displaying first UI objects (130) superimposed on an environment (150), the head-wearable electronic device (100) may detect a gaze of a user (120) toward another wearable device (110) positioned within the environment (150) viewed through the display assembly (208) and worn by the user (120) of the head-wearable electronic device (100) using images acquired through one or more first cameras (209). For example, the one or more first cameras (209) may be configured to acquire images of the eyes of the user (120) of the head-wearable electronic device (100). For example, the head-wearable electronic device (100) may use the images to identify a destination of the gaze of the user (120).

[0056] In operation 330, the head-wearable electronic device (100) may, based on the detection, display a user interface (e.g., a user interface (490) of FIG. 4) on the environment (150) through the display assembly (208), including second UI objects (e.g., second UI objects (430) of FIG. 4) and arranged in conjunction with the other wearable device (110). For example, the second UI objects (430) may include UI objects that each provide functions provided by the first UI objects (130). For example, the size of each of the second UI objects (430) may be smaller than the size of each of the first UI objects (130). For example, since the first UI objects (130) are displayed through the display assembly (208) as if they are located further from the user (120) than the second UI objects (430), the size of each of the second UI objects (430) may be smaller than the size of each of the first UI objects (130). However, the present invention is not limited thereto. For example, the size of each of the second UI objects (430) may be the same as the size of each of the first UI objects (130). For example, the size of each of the second UI objects (430) may be larger than the size of each of the first UI objects (130). For example, the user interface may be controlled according to user input received for the other wearable device (110). The user interface (490) including the second UI objects (430) is described and illustrated in more detail with reference to FIG. 4.

[0057] Referring to FIG. 4, a state (420) may be described as a state in which a user interface (490) including second UI objects (430) is displayed through the display assembly (208). The user interface (490) may be arranged in relation to (or in conjunction with) another wearable device (110). For example, the second UI objects (430) may be arranged in a form surrounding the other wearable device (110). However, the present invention is not limited thereto. For example, the second UI objects (430) may be arranged in a row next to the other wearable device (110).

[0058] The second UI objects (430) may include a second UI object (430-1), a second UI object (430-2), a second UI object (430-3), a second UI object (430-4), a second UI object (430-5), and a second UI object (430-6). For example, the first UI object (130-1) may correspond to the second UI object (430-1). For example, the first UI object (130-2) may correspond to the second UI object (430-2). For example, the first UI object (130-3) may correspond to the second UI object (430-3). For example, the first UI object (130-4) may correspond to the second UI object (430-4). For example, the first UI object (130-5) may correspond to the second UI object (430-5). For example, the first UI object (130-6) may correspond to the second UI object (430-6). However, the present invention is not limited thereto. For example, the number of second UI objects (430) may be greater than the number of first UI objects (130). For example, the number of second UI objects (430) may be less than the number of first UI objects (130). For example, at least one of the second UI objects (430) may provide a different function from the function provided by each of the first UI objects (130).

[0059] For example, the head-wearable electronic device (100) may display UI objects in relation to another wearable device (110) through the display assembly (208) based on content displayed through the display assembly (208) of the head-wearable electronic device (100). For example, while the head-wearable electronic device (100) outputs multimedia content through the display assembly (208) and a speaker (not shown), the head-wearable electronic device (100) may detect a gaze of a user (120) toward another wearable device (110), thereby displaying UI objects related to the content and arranged in relation to (or in conjunction with) the other wearable device (110). For example, the UI objects may include second UI objects (430). For example, the UI objects may include a UI object for controlling volume, a UI object for controlling playback of content (e.g., FF (fast forward), RW (rewind), STOP, Play, etc.), and a UI object for playing other content (e.g., playing the next content, skipping, etc.).

[0060] Although the second UI objects (430) are illustrated as being arranged in relation to another wearable device (110) through FIG. 4, the present disclosure is not limited thereto. For example, the head-wearable electronic device (100) may display a visual object (not shown) that was displayed on the display of another wearable device (110) through the display assembly (208). For example, the head-wearable electronic device (100) may display the visual object by overlapping the display of the other wearable device (110) by identifying the other wearable device (110) as shown through the display assembly (208). For example, while the head-mounted electronic device (100) may display the visual object by overlapping the display of another wearable device (110) via the display assembly (208), the other wearable device (110) may display a visual object different from the visual object on the display of the other wearable device (110). The visual object and the other visual object may include a visual object that was displayed on the display of the other wearable device (110). For example, the visual object may include a visual object used in the other wearable device (110).

[0061] For example, the head-wearable electronic device (100) can control the visual object by recognizing a gesture of a user (120) related to the display of another wearable device (110) shown through the display assembly (208). For example, the head-wearable electronic device (100) can move the visual object displayed through the display assembly (208) from a first position to a second position by recognizing a swiping gesture. However, the present invention is not limited thereto. For example, the head-wearable electronic device (100) can transmit information about the visual object to the other wearable device (110) through the communication circuit (205). For example, the head-wearable electronic device (100) can be connected to the other wearable device (110) through a Bluetooth communication technique. For example, another wearable device (110) may display a visual object on a display of the other wearable device (110) based on information about the visual object received from the head-mounted electronic device (100).

[0062] While the head-wearable electronic device (100) is operating in the first mode, the head-wearable electronic device (100) may change the screen provided through the display assembly (208) of the head-wearable electronic device (100) to a home screen by receiving a menu user input. For example, the home screen may include an interface including UI objects. For example, the home screen may include a screen that is first displayed through the display assembly (208) when the head-wearable electronic device (100) is executed. For example, the menu user input may be described as a user input for displaying a home screen through the display assembly (208) of the head-wearable electronic device (100). The home screen may be referred to as a default screen or an initial menu. The menu user input may include a user input using a button included in the head-wearable electronic device (100). The above menu user input may include identifying a gesture of the user (120) using second images acquired using one or more second cameras (210), but is not limited thereto. For example, while the head-wearable electronic device (100) is operating in the first mode, the head-wearable electronic device (100) may change the screen displayed through the display assembly (208) of the head-wearable electronic device (100) from the home screen to another screen by receiving the menu user input. For example, the other screen may include a visual object corresponding to a software application for a message. For example, the other screen may include a visual object corresponding to a software application for a video.

[0063] The user interface (490) including the second UI objects (430) can be controlled by user input received from another wearable device (110). For example, in response to the user input, the second UI object (430-1) can be moved from a first location to a second location. For example, the head-mounted electronic device (100) can execute a software application corresponding to the second UI object (430-1) in response to the user input. The user input is described and exemplified in more detail with reference to FIG. 5.

[0064] Figure 5 illustrates an example of controlling a user interface in response to user input.

[0065] Referring to FIG. 5, the head-wearable electronic device (100) can display a user interface (490) through the display assembly (208). A state (510) can be described as a state in which a user interface (490) arranged in relation to (or in conjunction with) another wearable device (110) is displayed through the display assembly (208). For example, an area (511) can be described as an area in which one of the second UI objects (430) can be positioned within the user interface (490). For example, the second UI objects (430-1) can be positioned in the area (511). For example, an area (512) can be described as an area in which one of the second UI objects can be positioned within the user interface. For example, the second UI objects (430-2) can be positioned in the area (512). For example, area (513) may be described as an area within the user interface (490) where one of the second UI objects (430) may be positioned. For example, the second UI objects (430-6) may be positioned in area (513).

[0066] The head-wearable electronic device (100) may, while displaying the user interface (490) arranged in relation to (or in conjunction with) another wearable device (110), receive, through the communication circuit (205), from the other wearable device (110), information about the user input received for the other wearable device (110). The head-wearable electronic device (100) may, using the received information, determine the user input detected by the other wearable device (110) as a user input received for the user interface (490). For example, the head-wearable electronic device (100) may recognize the user input received for the other wearable device (110) as a user input received for the user interface (490). For example, the user input may include a touch input for the other wearable device (110). For example, the user input may include, but is not limited to, input associated with a rotatable input device (e.g., input device (1030) of FIG. 10).

[0067] The head-wearable electronic device (100) may move the positions of the second UI objects (430) in response to a user input. For example, the head-wearable electronic device (100) may move the second UI object (430-1) located in the area (511) to the area (512). For example, the head-wearable electronic device (100) may position the second UI object (430-1) in the area (512). For example, the user input may include a user input for rotating the rotatable input device to the right. For example, the user input may include a user input for rotating the rotatable input device in a clockwise direction. For example, the user input may include a user input for swiping in a right direction on a display of another wearable device (110). For example, the head-wearable electronic device (100) can display a second UI object (430-1) located in an area (512) through the display assembly (208). For example, in response to the user input, the head-wearable electronic device (100) can slide the second UI object (430-1) from the area (511) to the area (512).

[0068] The head-wearable electronic device (100) may move the positions of the second UI objects (430) in response to a user input. For example, the head-wearable electronic device (100) may move the second UI object (430-1) located in the area (511) to the area (513). For example, the head-wearable electronic device (100) may position the second UI object (430-1) in the area (513). For example, the user input may include a user input for rotating the rotatable input device to the left. For example, the user input may include a user input for rotating the rotatable input device counterclockwise. For example, the user input may include a user input for swiping in the left direction on the display of another wearable device (110). For example, the head-wearable electronic device (100) may move the second UI objects (430) included in the user interface (490) in response to the user input. For example, the head-wearable electronic device (100) may move the second UI objects (430) clockwise or counterclockwise in response to the user input. For example, when the user (120) rotates the rotatable input device counterclockwise, the head-wearable electronic device (100) may rotate the second UI objects (430) counterclockwise. For example, when the user (120) rotates the rotatable input device clockwise, the head-wearable electronic device (100) may rotate the second UI objects (430) clockwise. For example, the head-wearable electronic device (100) may display a second UI object (430-1) located in an area (513) through the display assembly (208). For example, in response to the user input, the head-wearable electronic device (100) may slide the second UI object (430-1) from the area (511) to the area (513).For example, the head-wearable electronic device (100) may slide the second UI object (430-2) from the area (512) to the area (511) in response to the user input. The movement of the second UI object (430-1) and the second UI object (430-2) is described, but is not limited thereto. For example, the head-wearable electronic device (100) may slide the second UI objects (430) including the second UI object (430-3), the second UI object (430-4), the second UI object (430-5), and the second UI object (430-6).

[0069] The head-wearable electronic device (100) can control first UI objects (130) or second UI objects (430) in response to detection of a gaze of a user (120). The head-wearable electronic device (100) can detect the gaze of the user (120) using images acquired through one or more first cameras (209). The head-wearable electronic device (100) can control the user interface (490) based on the detection of the gaze. For example, based on detection of a gaze toward a second UI object (430-1), the head-wearable electronic device (100) can display a visual object (not shown) indicating that the second UI object (430-1) is selected through the display assembly (208). For example, the selected second UI object (430-1) can be visually highlighted. For example, the color of the selected second UI object (430-1) may change. For example, the head-mounted electronic device (100) may execute a software application corresponding to the selected second UI object (430-1) by receiving a user input.

[0070] The head-wearable electronic device (100) can recognize user input received for another wearable device (110) as user input received for the user interface (490). For example, the head-wearable electronic device (100) can transmit a signal to the other wearable device (110) to activate a second mode in which the user interface (490) is controlled according to the user input received for the other wearable device (110). Activation of the second mode is described and illustrated in more detail with reference to FIG. 6.

[0071] FIG. 6 is a flowchart illustrating an exemplary method for activating a mode in which a user interface is controlled based on user input received for another wearable device. This method may be executed by the head-mounted electronic device (100) illustrated in FIG. 2 or by at least one processor (207) of the head-mounted electronic device (100).

[0072] Referring to FIG. 6, in operation 610, the head-wearable electronic device (100) may display a visual affordance (e.g., visual affordance (730) of FIG. 7A) corresponding to another wearable device (110) as an overlay on the environment (150) surrounding the head-wearable electronic device (100) shown through the display assembly (208).

[0073] In operation 620, the head-wearable electronic device (100) may detect a user's gaze toward the visual affordance (e.g., the visual affordance (730) of FIG. 7A) displayed through the display assembly (208) using images acquired through one or more first cameras (209).

[0074] The head-wearable electronic device (100) can be connected to another wearable device (110). For example, the head-wearable electronic device (100) can be wirelessly connected to another wearable device (110). The head-wearable electronic device (100) and another wearable device (110) can be connected independently of the operation of FIG. 6. The embodiment is not limited thereto. For example, the head-wearable electronic device (100) can be connected to another wearable device (110) via a Bluetooth communication technique. By being connected to another wearable device (110), the head-wearable electronic device (100) can transmit information or signals to the other wearable device (110) via the communication circuit (205). A head-mounted electronic device (100) may be connected to another wearable device (110) and receive information or signals from the other wearable device (110) via a communication circuit (205). For example, the information may include information about user input. For example, the information may include information about notifications. For example, the information may include information about software applications. While the information included in the information is described, this is merely exemplary.

[0075] In operation 630, the head-wearable electronic device (100) may transmit a signal to the other wearable device (110) through the communication circuit (205) based on the detection to activate a second mode in which the user interface (490) is controlled according to a user input received from the other wearable device (110). For example, the signal may include a signal to activate a fourth mode. For example, the head-wearable electronic device (100) may activate the second mode by receiving a signal from the other wearable device (110) through the communication circuit (205) indicating that the mode of the other wearable device (110) is changed from the third mode to the fourth mode. For example, the second mode of the head-wearable electronic device (100) and the fourth mode of the other wearable device (110) may be activated simultaneously. For example, the head-wearable electronic device (100) may transmit a signal to the other wearable device (110) for activating a fourth mode in which the user interface (490) is controlled based on a user input received for the other wearable device (110) through the communication circuit (205) based on the detection. For example, while the head-wearable electronic device (100) is operating in the second mode, the other wearable device (110) may be operating in the fourth mode. For example, the head-wearable electronic device (100) may transmit the signal to activate the fourth mode to the other wearable device (110) based on being connected to the other wearable device (110). For example, the head-wearable electronic device (100) may transmit the signal to activate the fourth mode to the other wearable device (110) by being connected to the other wearable device (110).

[0076] For example, the head-wearable electronic device (100) may change the mode of the head-wearable electronic device (100) from the first mode to the second mode by transmitting a signal to the other wearable device (110) for activating the second mode and then receiving a signal from the other wearable device (110) indicating that the mode of the other wearable device (110) is changed from the third mode to the fourth mode. However, the present invention is not limited thereto. For example, the head-wearable electronic device (100) may change the mode of the head-wearable electronic device (100) from the first mode to the second mode by transmitting a signal to the other wearable device (110) indicating that the second mode is activated. For example, while the head-wearable electronic device (100) operates in the second mode, the other wearable device (110) may operate in the third mode.

[0077] Activation of the second mode is described and illustrated in more detail with reference to FIGS. 7a and 7b.

[0078] FIGS. 7A and 7B illustrate examples of a method for activating a mode in which the user interface is controlled based on user input received for the other wearable device.

[0079] Referring to FIG. 7A, a state (710) can be described as a state in which a visual affordance (730) is displayed through a display assembly (208). A head-wearable electronic device (100) can detect a user's gaze toward the visual affordance (730) using images acquired through one or more first cameras (209). The visual affordance (730) can correspond to another wearable device (110). For example, the visual affordance (730) can include a watch shape. However, the present invention is not limited thereto. By detecting the gaze, the head-wearable electronic device (100) can transmit a signal to the other wearable device (110) to activate a second mode through the communication circuit (205). For example, the head-wearable electronic device (100) can switch the mode of the head-wearable electronic device (100) from the first mode to the second mode by transmitting the signal to another wearable device (110). For example, the other wearable device (110) can switch the mode of the other wearable device (110) from the third mode to the fourth mode by receiving the signal. For example, while the head-wearable electronic device (100) is operating in the second mode, the other wearable device (110) can be operated in the fourth mode.

[0080] Another wearable device (110) may be positioned within an environment (150) viewed through a display assembly (208) of a head-wearable electronic device (100). The head-wearable electronic device (100) may detect a user's gaze toward the other wearable device (110) viewed through the display assembly (208) using images acquired through one or more first cameras (209). By detecting the gaze, the head-wearable electronic device (100) may transmit a signal to the other wearable device (110) through the communication circuit (205) to activate a second mode.

[0081] The head-mounted electronic device (100) can transmit a signal to activate the second mode using a QR (quick response) code (740). State (720) can be described as a state in which a QR code (740) is displayed on the display of another wearable device (110). The other wearable device (110) can display the QR code (740) on the display of the other wearable device (110). For example, the other wearable device (110) can display the QR code (740) on the display of the other wearable device (110) by receiving a user input that causes the display of the QR code (740). For example, the user input can include a user input for changing the mode of the other wearable device (110) from the third mode to the fourth mode. For example, the user input may include a user input for changing the mode of the head-wearable electronic device (100) from a first mode to a second mode. For example, the user input may include a touch input. For example, the user input may include an input using an input device (e.g., the input device (1030) of FIG. 10). The head-wearable electronic device (100) may recognize a QR code (740) displayed on a display of another wearable device (110) using one or more second cameras (210). By recognizing the QR code (740), the head-wearable electronic device (100) may transmit a signal to the other wearable device (110) through the communication circuit (205) to activate the second mode. For example, the QR code (740) may include information that causes the mode of the head-wearable electronic device (100) to be switched from a first mode to a second mode. For example, the QR code (740) may include information for pairing the head-wearable electronic device (100) with another wearable device (110).For example, the QR code (740) may include information for switching the mode of another wearable device (110) from the third mode to the fourth mode.

[0082] A head-wearable electronic device (100) can transmit a signal to another wearable device (110) via a communication circuit (205) to activate a second mode by identifying a gesture of a user (120). The head-wearable electronic device (100) can include one or more second cameras (210). The one or more second cameras (210) can be configured to acquire images of an environment (150) surrounding the head-wearable electronic device (100). The head-wearable electronic device (100) can identify a gesture associated with the other wearable device using the second images acquired through the one or more second cameras. For example, the user (120) can wear the other wearable device (110) on his or her hand (or wrist). For example, the gesture may include a gesture using the hand of a user (120) wearing another wearable device (110). For example, the gesture may include a pinch. For example, the gesture may include a fist-closing and fist-opening motion with the hand wearing another wearable device (110). For example, the gesture may include a fist-closing and fist-opening motion repeated twice with the hand wearing another wearable device (110). However, the present invention is not limited thereto. Based on the identification, the head-mounted electronic device (100) may transmit a signal to the other wearable device (110) to activate a second mode in which the user interface (490) is controlled according to a user input received for the other wearable device (110) through the communication circuit (205). For example, while a head-mounted electronic device (100) is operating in a second mode, another wearable device (110) may be operating in a fourth mode.

[0083] For example, the head-wearable electronic device (100) can change a mode of the head-wearable electronic device (100) by receiving a set of user inputs. For example, the set of user inputs can combine at least two of identifying a gesture, receiving an input via an input device (e.g., the input device (1030) of FIG. 10), receiving a touch input, and identifying a voice of the user (120). For example, the head-wearable electronic device (100) can change the mode of the head-wearable electronic device (100) from a first mode to a second mode by identifying a gesture of the user (120) using second images acquired via one or more second cameras (210), and then receiving a user input via the input device (e.g., the input device (1030) of FIG. 10). For example, the head-wearable electronic device (100) can activate the second mode by identifying the user's (120) gesture using one or more second cameras (210) and then identifying the user's (120) voice. For example, based on the head-wearable electronic device (100) activating the second mode, another wearable device (110) can activate the fourth mode. However, the present invention is not limited thereto. For example, the head-wearable electronic device (100) can change the mode of the head-wearable electronic device (100) from the second mode to the first mode by identifying a gesture of making a fist.

[0084] Another wearable device (110) can transmit a signal to the head-wearable electronic device (100) to activate a second mode of the head-wearable electronic device (100) via a communication circuit (not shown) of the other wearable device (110) (e.g., a communication module (1490) of FIG. 14). The head-wearable electronic device (100) can receive a signal to activate the second mode from the other wearable device (110) via the communication circuit (205). For example, based on the reception, the head-wearable electronic device (100) can display a window (not shown) to inquire whether to activate the second mode via the display assembly (208). For example, the window can include a visual object (not shown) to activate the second mode and a visual object (not shown) to maintain the first mode. For example, the head-wearable electronic device (100) may receive a user input related to the second mode while displaying the window. For example, the head-wearable electronic device (100) may receive a user input that causes the activation of the second mode while displaying the window. For example, the user input may include a touch input. For example, the user input may include a swipe input. For example, the head-wearable electronic device (100) may activate the second mode in response to the user input. For example, the head-wearable electronic device (100) may switch the mode from the first mode to the second mode. However, the present invention is not limited thereto. For example, the head-wearable electronic device (100) may activate the second mode by receiving a signal for activating the second mode from another wearable device (110).

[0085] Referring to FIG. 7B, a state (750) may be described as a state of controlling a user interface (780) displayed through a display assembly (208) of a head-wearable electronic device (100) using external electronic devices. For example, the head-wearable electronic device (100) may display visual affordances corresponding to each of a plurality of external electronic devices through the display assembly (208). For example, the visual affordances may include visual affordance (760) and visual affordance (770). For example, visual affordance (760) may correspond to a first electronic device (765). For example, visual affordance (770) may correspond to a second electronic device (e.g., another wearable device (110)). For example, the first electronic device (765) may include a smartphone. For example, the second electronic device may include a smartwatch. The head-wearable electronic device (100) can display a user interface (780) corresponding to a software application running on the head-wearable electronic device (100) through the display assembly (208). While displaying the user interface (780), the head-wearable electronic device (100) can display a visual affordance (760) and a visual affordance (770) through the display assembly (208). The head-wearable electronic device (100) can detect a gaze of a user (120) toward the visual affordance (760) or the visual affordance (770) using second images acquired through one or more second cameras (210). For example, the head-wearable electronic device (100) can change a mode of the head-wearable electronic device (100) from a first mode to a second mode based on detecting the gaze of the user (120).For example, the head-wearable electronic device (100) can control the user interface (780) based on a user input included in information received from the first electronic device (765) by detecting the gaze of the user (120) toward the visual affordance (760). However, the present invention is not limited thereto. The head-wearable electronic device (100) can change the mode of the head-wearable electronic device (100) from the first mode to the second mode based on a user input selecting the visual affordance (760) or the visual affordance (770). For example, the user input can include a gesture. The head-wearable electronic device (100) can control the user interface (780) based on the user input included in information received from the first electronic device (765) by identifying a gesture selecting the visual affordance (760). For example, the user interface (780) may include a user interface of a software application for a message. For example, by detecting the gaze of the user (120) toward the visual affordance (760) corresponding to the first electronic device (765), the head-wearable electronic device (100) may receive information for controlling the user interface (780) from the first electronic device (765) via the communication circuit (205). For example, the first electronic device (765) may display a keyboard for entering a message on the display of the first electronic device (765) to control the software application for the message. For example, the first electronic device (765) may transmit information including a user input for the keyboard to the head-wearable electronic device (100). For example, the head-wearable electronic device (100) may perform an action corresponding to the user input by receiving the information.

[0086] The user (120) may perceive the size of the user interface (780) provided through the display assembly (208) of the head-wearable electronic device (100) to be larger than the size of the user interface displayed on the display of the first electronic device (765). For example, since the size of the user interface (780) provided by the head-wearable electronic device (100) is perceived to be larger than the size of the user interface displayed on the display of the first electronic device (765), the head-wearable electronic device (100) may replace the display of the first electronic device (765). For example, the head-wearable electronic device (100) may provide the user interface (780) through the display assembly (208) while receiving information including a user input for the keyboard from the first electronic device (765) through the communication circuit (205). For example, since the keyboard displayed on the display of the first electronic device (765) is used through the user's (120) fingers, the first electronic device (765) can be used to control the head-wearable electronic device (100). For example, since the amount of content that can be displayed on the display of the first electronic device (765) is smaller than the amount of content that can be provided through the display assembly (208) of the head-wearable electronic device (100), it may be required to determine the range of controllable user interfaces (780) through the first electronic device (765). For example, the head-wearable electronic device (100) can determine the range of controllable user interfaces (780) using the first electronic device (765) depending on the state of the first electronic device (765). For example, the head-mounted electronic device (100) can determine the user interface (780) controlled by the first electronic device (765) based on the size of the display of the first electronic device (765).For example, the head-mounted electronic device (100) can determine an area to be controlled by information received from the first electronic device (765) based on the size of the display of the first electronic device (765).

[0087] The head-wearable electronic device (100) can display a user interface corresponding to the user interface (780) through the display assembly (208) based on the characteristics of the first electronic device (765). For example, the head-wearable electronic device (100) can display a user interface for the user interface (780) through the display assembly (208) based on the capabilities of the first electronic device (765). For example, the capabilities of the first electronic device (765) can be determined according to the size of the display of the first electronic device (765). For example, the capabilities of the first electronic device (765) can be determined according to the size of an area of ​​the first electronic device (765) that allows touch input. For example, the capabilities can be determined according to the characteristics of the device. For example, the capability may be determined according to an input device (e.g., a rotatable input device, a touchpad, a touchscreen) included in the first electronic device (765). For example, the capability may be determined according to a type (e.g., a smartwatch, a smartphone, a laptop computer, a desktop computer) of the first electronic device (765). For example, the head-wearable electronic device (100) may display a user interface adaptive to the first electronic device (765) through the display assembly (208) according to the capability of the first electronic device (765). For example, the head-wearable electronic device (100) may identify the capability of the first electronic device (765) using an image acquired through one or more second cameras (210). For example, the head-mounted electronic device (100) can receive information about the capabilities of the first electronic device (765) from the first electronic device (765) via the communication circuit (205).For example, the head-wearable electronic device (100) can display another user interface for controlling the user interface (780) through the display assembly (208) in relation to the first electronic device (765) based on the capabilities of the first electronic device (765). For example, when the capabilities of the first electronic device (765) are large, the head-wearable electronic device (100) can display a user interface (e.g., a keyboard) capable of controlling the user interface (780) through the display assembly (208) in relation to the first electronic device (765). For example, the head-wearable electronic device (100) can display a user interface capable of controlling the user interface (780) in relation to the first electronic device (765) being viewed through the display assembly (208). For example, the head-wearable electronic device (100) may control a user interface (780) based on receiving user input for a user interface displayed in relation to the first electronic device (765). For example, the head-wearable electronic device (100) may transmit a message based on receiving user input for composing text.

[0088] When the capability of the first electronic device (765) is small (e.g., when the length of the display of the first electronic device (765) is less than 4 cm), the head-wearable electronic device (100) can display a user interface (e.g., a visual object for direction) that can control the user interface (780) in a limited manner in relation to the first electronic device (765) through the display assembly (208). For example, the head-wearable electronic device (100) can display a user interface that can control the user interface (780) in relation to the first electronic device (765) being viewed through the display assembly (208). For example, the head-wearable electronic device (100) can control the user interface (780) based on receiving a user input for the user interface displayed in relation to the first electronic device (765). For example, the head-wearable electronic device (100) may scroll a list of messages based on receiving a user input for scrolling within the user interface (780). For example, the head-wearable electronic device (100) may display an adaptive user interface through the display assembly (208) in relation to the first electronic device (765) based on the capabilities of the first electronic device (765). For example, if the capabilities of the first electronic device (765) are limited, the functions that can be executed through the user interface displayed in relation to the first electronic device (765) may be limited. For example, because the capabilities of the first electronic device (765) are limited, the head-wearable electronic device (100) may not be able to display a user interface for writing text in relation to the first electronic device (765). For example, the head-mounted electronic device (100) may display the adaptive user interface via the display assembly (208) overlapping the first electronic device (765), but is not limited thereto.For example, a head-mounted electronic device (100) may display the adaptive user interface through a display assembly (208) around the first electronic device (765).

[0089] Referring back to FIG. 6, at operation 640, the other wearable device (110) may receive a signal from the head-wearable electronic device (100) to activate a second mode. For example, the other wearable device (110) may receive a signal from the head-wearable electronic device (100) to activate a fourth mode. For example, while the head-wearable electronic device (100) is operating in the second mode, the other wearable device (110) may be operating in the fourth mode. For example, the other wearable device (110) may display a window (not shown) asking whether to activate the mode on a display (not shown) of the other wearable device (110) (e.g., the display module (1460) of FIG. 14). For example, the other wearable device (110) may switch the mode of the other wearable device (110) from the third mode to the fourth mode based on receiving a signal from the head-mounted electronic device (100) to activate the second mode.

[0090] In operation 645, for example, while the other wearable device (110) displays the window, the other wearable device (110) may receive a user input for activating the second mode. For example, the other wearable device (110) may activate the fourth mode in response to the user input. For example, while the other wearable device (110) displays the window, the other wearable device (110) may receive a user input for activating the fourth mode. For example, the other wearable device (110) may activate the fourth mode in response to the user input. For example, while the other wearable device (110) activates the fourth mode, the head-wearable electronic device (100) may activate the second mode. For example, when the other wearable device (110) activates the fourth mode, the head-wearable electronic device (100) may activate the second mode. However, the present invention is not limited thereto. The other wearable device (110) may activate the fourth mode by receiving a signal for activating the fourth mode from the head-wearable electronic device (100). For example, while the head-wearable electronic device (100) is operating in the second mode, the other wearable device (110) may be operating in the fourth mode. For example, the other wearable device (110) may skip operations 640 and 645 because it may activate the fourth mode by receiving a signal for activating the fourth mode from the head-wearable electronic device (100). For example, the other wearable device (110) may bypass, omit, or refrain from displaying a window inquiring about activating the fourth mode. For example, the other wearable device (110) may bypass, omit, or refrain from receiving a user input for activating the fourth mode.

[0091] For example, the other wearable device (110) may refrain from displaying a first notification object (not shown) on the display of the other wearable device (110) while the fourth mode is activated. For example, the notification object may be described as an object indicating that information (e.g., an alarm or a message) has been received. For example, the other wearable device (110) may refrain from displaying a notification object on the display of the other wearable device (110) while the fourth mode is activated. For example, the other wearable device (110) may transmit data to the head-mounted electronic device (100) for displaying a notification object on the display of the other wearable device (110) while the fourth mode is activated. For example, the data may include information about an alarm. For example, the data may include information about an email. For example, the data may include information about a call. For example, the head-wearable electronic device (100) may receive the data and display a second notification object (not shown) corresponding to the data through the display assembly (208) using the data. For example, another wearable device (110) may be displayed through the display assembly (208) of the head-wearable electronic device (100). However, the present invention is not limited thereto. For example, while the other wearable device (110) is displayed through the display assembly (208) of the head-wearable electronic device (100), the first notification object (not shown) may be displayed on the display of the other wearable device (110). For example, while the other wearable device (110) is located outside the environment (150) viewed through the display assembly (208) of the head-mounted electronic device (100), the other wearable device (110) may stop or refrain from displaying the first notification object on the display of the other wearable device (110).

[0092] For example, the other wearable device (110) may reduce power consumption for displaying the first notification object by refraining from displaying the first notification object on the display of the other wearable device (110). For example, the sensitivity of touch input obtained through the display of the other wearable device (110) may be increased by refraining from displaying the first notification object on the display of the other wearable device (110). For example, the other wearable device (110) may transmit information to the head-mounted electronic device (100) while the fourth mode is activated. For example, the information may include information related to an email. For example, the information may include information related to a reminder. For example, the information may include information related to a call. However, the present invention is not limited thereto.

[0093] In operation 650, the other wearable device (110) may transmit a signal to the head-wearable electronic device (100) to activate the second mode (or a signal to switch to the second mode). The embodiment is not limited thereto, and the other wearable device (110) may transmit a signal to the head-wearable electronic device (100) indicating that the other wearable device (110) has entered the fourth mode. For example, while the head-wearable electronic device (100) is operating in the second mode, the other wearable device (110) may be operating in the fourth mode. For example, the head-wearable electronic device (100) may receive a signal to activate the second mode from the other wearable device (110) via the communication circuit (205). For example, the head-wearable electronic device (100) may activate the second mode by receiving the signal. However, the embodiment is not limited thereto. For example, the head-wearable electronic device (100) can activate the second mode by transmitting a signal to the other wearable device (110) via the communication circuit (205) to activate the second mode for controlling the user interface (490) by user input received from the other wearable device (110). For example, the head-wearable electronic device (100) can activate the second mode by transmitting a signal to the other wearable device (110) via the communication circuit (205) to activate the second mode for controlling the user interface (490) by user input received from the other wearable device (110), thereby causing the other wearable device (110) to activate the fourth mode.

[0094] In operation 670, the head-wearable electronic device (100) may detect a gaze toward another wearable device (110) positioned within the environment (150) and worn by a user (120) of the head-wearable electronic device (100) through images acquired through one or more first cameras (209) while displaying first UI objects (130) overlaid on the environment (150) as viewed through the display assembly (208). Operation 670 may correspond to operation 320.

[0095] At operation 680, the head-wearable electronic device (100) may stop displaying first UI objects (130) being displayed through the display assembly (208) based on detecting a gaze toward another wearable device, and may display a user interface including second UI objects (430) arranged in relation to (or in conjunction with) another wearable device (110) on the environment (150) through the display assembly (208). Operation 680 may correspond to operation 330.

[0096] The head-wearable electronic device (100) can deactivate a second mode in which the user interface (490) is controlled based on user input received from another wearable device (110). For example, the head-wearable electronic device (100) can switch the mode from the second mode to the first mode. For example, the head-wearable electronic device (100) can transmit a signal related to the deactivation of the second mode to the other wearable device (110) via the communication circuit (205). The deactivation of the second mode is described and exemplified in more detail with reference to FIG. 8.

[0097] FIG. 8 illustrates an example of a method for disabling a mode in which the user interface is controlled based on user input received for the other wearable device.

[0098] Referring to FIG. 8, a state (810) may be described as a state in which another visual affordance (830) is displayed through the display assembly (208). For example, the other visual affordance (830) may correspond to another wearable device (110). For example, the function of the other visual affordance (830) may be different from the function of the visual affordance (730). For example, the visual affordance (730) may be displayed through the display assembly (208) to activate a second mode. For example, the other visual affordance (830) may be displayed through the display assembly (208) to deactivate a second mode. For example, the other visual affordance (830) may be displayed through the display assembly (208) to activate a first mode. For example, the shape of the other visual affordance (830) may be different from the shape of the visual affordance (730). However, this is not limited thereto. For example, the shape of the other visual affordance (830) may be the same as the shape of the visual affordance (730). While the second mode is activated, the head-wearable electronic device (100) may detect a gaze toward the other visual affordance (830) displayed through the display assembly (208) using images acquired through one or more first cameras (209). The head-wearable electronic device (100) may deactivate the second mode based on detecting a gaze toward the other visual affordance (830).

[0099] The head-wearable electronic device (100) can detect a gaze using images acquired through one or more first cameras (209) to transmit another signal to another wearable device (110) to deactivate the second mode. State (820) can be described as a state in which the first UI objects (130) are displayed through the display assembly (208) while the second mode is activated. For example, the head-wearable electronic device (100) can display the first UI objects (130) and the second UI objects (430) through the display assembly (208) while the second mode is activated. For example, the head-wearable electronic device (100) can detect a user's gaze toward the first UI objects (130) displayed through the display assembly (208) using images acquired through one or more first cameras (209) while the second mode is activated. For example, the head-mounted electronic device (100) can detect a gaze toward an external object located within an environment (150) viewed through the display assembly (208) using images acquired through one or more first cameras (209) while the second mode is activated.

[0100] Based on the detection, the head-wearable electronic device (100) can transmit another signal to the other wearable device (110) via the communication circuit (205) to deactivate a second mode in which the user interface (490) is controlled according to a user input received from the other wearable device. The other wearable device (110) can receive the other signal from the head-wearable electronic device (100). The head-wearable electronic device (100) can deactivate the second mode by transmitting the other signal to the other wearable device (110). After the second mode is deactivated, the head-wearable electronic device (100) can reactivate the second mode by detecting a gaze toward a visual affordance (730) displayed through the display assembly (208) using images acquired through one or more first cameras (209). The head-wearable electronic device (100) can receive information including a user input from another wearable device (110) through the communication circuit (205) while the second mode is activated. The head-wearable electronic device (100) can move the position of second UI objects (430) displayed through the display assembly (208) based on the user input. The head-wearable electronic device (100) can accept or process a user input included in the information received from the other wearable device (110) while the second mode is activated, and a user input included in the information received from the other wearable device (110) among the user inputs received by the head-wearable electronic device (100). For example, the head-wearable electronic device (100) may perform an action based on user input included in information received from another wearable device (110) and user input included in information received from another wearable device (110) while the second mode is activated.The authorization of the above user input is described and illustrated in more detail with reference to FIG. 9.

[0101] FIG. 9 is a flowchart illustrating an exemplary method for authorizing received user input for another wearable device. This method may be executed by the head-mounted electronic device (100) illustrated in FIG. 2 or by at least one processor (207) of the head-mounted electronic device (100).

[0102] Referring to FIG. 9, in operation 910, a head-mounted electronic device (100) may detect a user's gaze toward the user interface (e.g., user interface (490)) using images acquired through one or more first cameras (209) while displaying a user interface (e.g., user interface (490)) arranged in relation to (or in conjunction with) another wearable device (110).

[0103] In operation 920, the head-mounted electronic device (100) may detect the gaze while displaying the user interface arranged in relation to (or in conjunction with) another wearable device (110), and then receive information about user input received for the other wearable device (110) from the other wearable device (110) through the communication circuit (205).

[0104] In operation 930, the head-wearable electronic device (100) may, in response to the reception, apply the user input received for the other wearable device (110) among the user input received for the other wearable device (110) and the other user input for controlling the user interface (490) received for the head-wearable electronic device (100). The user input may be associated with the other wearable device (110). For example, the head-wearable electronic device (100) may ignore the other user input for controlling the user interface (490) received for the head-wearable electronic device (100). For example, the user input may include a touch input received through a display of the other wearable device (110). The user input is described and illustrated in more detail with reference to FIG. 10.

[0105] Figure 10 illustrates examples of user input received for different wearable devices.

[0106] Referring to FIG. 10, another wearable device (110) may transmit information including the user input to the head-mounted electronic device (100). State (1010) may be described as a state in which the other wearable device (110) detects a touch input. For example, the other wearable device (110) may receive a swipe gesture through the display of the other wearable device (110).

[0107] State (1020) may be described as a state in which another wearable device (110) detects an input to the input device. For example, the other wearable device (110) may include an input device (1030). For example, the input device (1030) may be positioned relative to a display of the other wearable device (110). For example, the input device (1030) may be rotatable. For example, the input device (1030) may include a bezel. For example, the input device (1030) may include a crown.

[0108] A head-wearable electronic device (100) can receive information including a user input from another wearable device (110) through a communication circuit (205). The head-wearable electronic device (100) can control a user interface (490) based on the user input. For example, the head-wearable electronic device (100) can move the position of second UI objects (430) included in the user interface (490). For example, the head-wearable electronic device (100) can execute a software application corresponding to one of the second UI objects (430) included in the user interface (490) (e.g., the second UI object (430-1)). However, the present invention is not limited thereto.

[0109] When the other wearable device (110) does not receive user input, the head-mounted electronic device (100) can control the user interface (490). For example, a state in which the other wearable device (110) does not receive user input may include a state in which the other wearable device (110) is deactivated. For example, a state in which the other wearable device (110) does not receive user input may include an off state. For example, a state in which the other wearable device (110) does not receive user input may include a power-off state. For example, a state in which the other wearable device (110) does not receive user input may include a turn-off state. For example, the deactivated state, the off state, the power-off state, and the turn-off state may include a state in which the battery of the other wearable device (110) is discharged.

[0110] For example, the head-wearable electronic device (100) may include one or more second cameras (210). The one or more second cameras (210) may be configured to acquire images of an environment (150) surrounding the head-wearable electronic device (100). The head-wearable electronic device (100) may use the images acquired through the one or more second cameras (210) to identify a gesture associated with another wearable device (110). Based on the identification, the head-wearable electronic device (100) may control the user interface (490). For example, the head-wearable electronic device (100) may move the second UI objects (430) or execute a software application corresponding to the second UI object (430-1) based on the identified gesture while the other wearable device (110) is in an inactive state. For example, the gesture may include a gesture of touching the display of another wearable device (110). For example, the gesture may include a motion of rotating the input device (1030). However, the present invention is not limited thereto.

[0111] The head-wearable electronic device (100) can receive user input related to the head-wearable electronic device (100) through the communication circuit (205) while the second mode for controlling the user interface (490) according to the user input received for the other wearable device (110) is activated. For example, the head-wearable electronic device (100) can authorize the user input received for the other wearable device (110) among the user input received for the other wearable device (110) and the other user input for controlling the user interface (490) received for the head-wearable electronic device (100). The authorization of the user input is described and illustrated in more detail with reference to FIG. 11.

[0112] Figure 11 illustrates examples of second UI objects moving in response to user input.

[0113] Referring to FIG. 11, a state (1110) may be described as a state in which a user interface is controlled by a user input included in information received from another wearable device (110) while a second mode is activated. Before the head-wearable electronic device (100) receives the information, the head-wearable electronic device (100) may display a user interface (490) through the display assembly (208). The user interface (490) may include second UI objects (430). The second UI objects (430) may include a second UI object (430-1), a second UI object (430-2), and a second UI object (430-3). For example, referring to state (1110), before the head-wearable electronic device (100) receives the information, the second UI object (430-2) may be located in an area (511). For example, before the head-wearable electronic device (100) receives the information, the second UI object (430-3) may be positioned in the area (512). The head-wearable electronic device (100) may move the position of the second UI object (430-2) in response to the user input. For example, the head-wearable electronic device (100) may move the second UI object (430-2) from the area (511) to the area (513) in response to the user input. For example, the area (511) may include the first position. For example, the area (513) may include the second position. For example, the head-wearable electronic device (100) may move the position of the second UI object (430-3) in response to the user input. For example, the head-mounted electronic device (100) may move the second UI object (430-3) from area (512) to area (511) in response to the user input.

[0114] The head-wearable electronic device (100) may receive another user input for controlling the user interface (490) while the head-wearable electronic device (100) displays the user interface (490) arranged in relation to (or in conjunction with) another wearable device (110) via the display assembly (208). For example, state (1120) may be described as a state in which the second mode is activated and the head-wearable electronic device (100) displays the user interface (490) arranged in relation to (or in conjunction with) another wearable device (110), while the head-wearable electronic device (100) receives another user input for controlling the user interface (490). For example, the head-wearable electronic device (100) may receive the other user input. For example, the other user input may include a touch input to the head-wearable electronic device (100). For example, the other user input may include an input to an input device of the head-wearable electronic device (100). The head-wearable electronic device (100) may display a user interface (490) via the display assembly (208). The user interface (490) may include second UI objects (430). For example, referring to state (1120), the second UI object (430-1) may be located in area (511). For example, the second UI object (430-2) may be located in area (512). For example, the second UI object (430-6) may be located in area (513). The head-wearable electronic device (100) may refrain from, block, or skip feedback on other user input while the head-wearable electronic device (100) displays the user interface (490) arranged in relation to (or in conjunction with) the other wearable device via the display assembly (208).For example, after the head-wearable electronic device (100) receives the other user input, the second UI object (430-1) may remain positioned in the area (511). However, the present invention is not limited thereto. The head-wearable electronic device (100) may display the second UI objects (430) through the display assembly (208) in response to the other user input. For example, the head-wearable electronic device (100) may position the second UI object (430-1) in the area (512) or the area (513) in response to the other user input.

[0115] The head-mounted electronic device (100) may provide a haptic notification in response to the user input. For example, the haptic notification may include vibration. The haptic notification is described and exemplified in more detail with reference to FIG. 12.

[0116] Figure 12 illustrates an example of providing a haptic notification in response to user input.

[0117] Referring to FIG. 12, a state (1210) may be described as a state of receiving information including a user input from another wearable device (110). The head-wearable electronic device (100) may control a user interface (490) in response to the user input. The head-wearable electronic device (100) may further include a vibration motor (e.g., an actuator) for haptic notification (e.g., a haptic module (1479) of FIG. 14). The head-wearable electronic device (100) may provide a haptic notification in response to the user input. The head-wearable electronic device (100) may further include a speaker (not shown) (e.g., an audio module (1470) of FIG. 14). The head-wearable electronic device (100) may output audio in response to the user input.

[0118] State (1220) can be described as a state in which the head-wearable electronic device (100) identifies a gesture associated with another wearable device (110). For example, the head-wearable electronic device (100) can identify a gesture associated with another wearable device (110) using images acquired through one or more second cameras (210). For example, the gesture can include a gesture of twice repeating an action of changing from a clenched fist state (1230) using a hand wearing the head-wearable electronic device (100) to an open hand state (1240) using the hand wearing the head-wearable electronic device (100). The head-wearable electronic device (100) can control a user interface (490) in response to the gesture. For example, the head-wearable electronic device (100) can move the positions of the second UI objects (430) included in the user interface (490) by identifying the gesture. For example, the head-wearable electronic device (100) can execute a software application corresponding to each of the second UI objects (430) included in the user interface (490) by identifying the gesture. The head-wearable electronic device (100) can provide a haptic notification in response to the gesture. The head-wearable electronic device (100) can output audio in response to the gesture. Although FIG. 12 illustrates the head-wearable electronic device (100) providing a haptic notification and outputting audio in response to the gesture, the embodiments of the present disclosure are not limited thereto. For example, another wearable device (110) can provide feedback in response to the gesture. For example, another wearable device (110) may provide a haptic notification in response to the gesture. For example, another wearable device (110) may output audio in response to the gesture.

[0119] For example, the user (120) may recognize that the user interface (490) is controlled through the haptic notification. For example, the user (120) may recognize that the user interface (490) is controlled through the output audio. For example, the user (120) may recognize that one of the second UI objects (430) is selected through the haptic notification. For example, the user (120) may recognize that one of the second UI objects (430) is selected through the output audio. For example, the user (120) may recognize that a software application corresponding to each of the second UI objects (430) is executed through the haptic notification. For example, the user (120) may recognize that a software application corresponding to each of the second UI objects (430) is executed through the output audio.

[0120] Figure 13 illustrates an example of an environment where the head-mounted electronic device is a video see-through (VST) device.

[0121] Referring to FIG. 13, the head-wearable electronic device (100) may be described as a video see-through (VST) device. When the head-wearable electronic device (100) is a VST device, the display assembly (208) may include at least one opaque display. For example, the head-wearable electronic device (100) may display an image representing at least a portion of an environment (150) positioned within a FOV of one or more second cameras (210) through the display assembly (208). For example, the image may represent at least a portion of the environment (150) by including external objects within the environment (150), such as a picture frame (160), a flower pot (170), and a drawer (180).

[0122] In one embodiment where the head-wearable electronic device (100) is a VST device, the head-wearable electronic device (100) can at least partially (e.g., completely) cover both eyes of a user wearing the head-wearable electronic device (100). The display assembly (208), when worn by the user (120), can display a screen that at least partially occupies the FOV of both eyes of the user (120). The head-wearable electronic device (100) can display images and / or videos from a camera positioned toward the external environment (e.g., one or more second cameras (210)) through the display assembly (208). By using images and / or videos displayed through the display assembly (208), the head-wearable electronic device (100) can provide a user experience such that the user (120) wearing the head-wearable electronic device (100) sees (directly) the external environment (through the head-wearable electronic device (100)), even though the head-wearable electronic device (100) covers both eyes of the user (120).

[0123] Referring to FIGS. 1 to 13, operations performed by a head-wearable electronic device (100) that is an AR glass are illustrated. However, this is for convenience of explanation. For example, the operations of the present disclosure may be performed by a head-wearable electronic device (100) that is a VST device. For example, when performed by a head-wearable electronic device (100) that is a VST device, another wearable device (110) that is included in the head-wearable electronic device (100) that is an AR glass and is shown through a display assembly (208) that includes at least one transparent display may be replaced with a visual object displayed through the display assembly (208) (e.g., a visual object representing another wearable device (110). For example, a visual object (1360) may correspond to a picture frame (160). For example, a visual object (1370) may correspond to a flower pot (170). For example, a visual object (1380) may correspond to a drawer (180).

[0124] For example, the head-wearable electronic device (100) may display, through the display assembly (208), first UI objects (130) on each of the images acquired through one or more second cameras (210). For example, while the head-wearable electronic device (100) displays the first UI objects (130) on each of the images, the head-wearable electronic device (100) may use the images acquired through one or more first cameras (209) to detect a gaze directed toward a visual affordance (730) within the image(s) that corresponds to another wearable device (110) positioned in an environment (150) around the head-wearable electronic device (100) and worn by a user (120) of the head-wearable electronic device (100). For example, the head-mounted electronic device (100) may, based on the detection, display a user interface (490) on the image via the display assembly (208), including second UI objects (430) and arranged in relation to (or in conjunction with) the visual object.

[0125] FIG. 14 is a block diagram of an electronic device within a network environment according to various embodiments.

[0126] Referring to FIG. 14, in a network environment (1400), an electronic device (1401) (e.g., a head-wearable electronic device (100) of FIG. 2) may communicate with an electronic device (1402) via a first network (1498) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (1404) or a server (1408) via a second network (1499) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (1401) may communicate with the electronic device (1404) via the server (1408). According to one embodiment, the electronic device (1401) may include a processor (1420) (e.g., at least one processor (207) of FIG. 2), a memory (1430) (e.g., the memory (206) of FIG. 2), an input module (1450), an audio output module (1455), a display module (1460) (e.g., the display assembly (208) of FIG. 2), an audio module (1470), a sensor module (1476), an interface (1477), a connection terminal (1478), a haptic module (1479), a camera module (1480) (e.g., one or more first cameras (209) and one or more second cameras (210) of FIG. 2), a power management module (1488), a battery (1489), a communication module (1490), a subscriber identification module (1496), or an antenna module (1497). In some embodiments, the electronic device (1401) may omit at least one of these components (e.g., the connection terminal (1478)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (1476), the camera module (1480), or the antenna module (1497)) may be integrated into a single component (e.g., the display module (1460)).

[0127] The processor (1420) may, for example, execute software (e.g., a program (1440)) to control at least one other component (e.g., a hardware or software component) of the electronic device (1401) connected to the processor (1420) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (1420) may store commands or data received from other components (e.g., a sensor module (1476) or a communication module (1490)) in a volatile memory (1432), process the commands or data stored in the volatile memory (1432), and store result data in a non-volatile memory (1434). According to one embodiment, the processor (1420) may include a main processor (1421) (e.g., a central processing unit or an application processor) or an auxiliary processor (1423) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (1421). For example, when the electronic device (1401) includes the main processor (1421) and the auxiliary processor (1423), the auxiliary processor (1423) may be configured to use less power than the main processor (1421) or to be specialized for a given function. The auxiliary processor (1423) may be implemented separately from the main processor (1421) or as a part thereof.

[0128] The auxiliary processor (1423) may control at least a portion of functions or states associated with at least one component (e.g., the display module (1460), the sensor module (1476), or the communication module (1490)) of the electronic device (1401), for example, on behalf of the main processor (1421) while the main processor (1421) is in an inactive (e.g., sleep) state, or together with the main processor (1421) while the main processor (1421) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (1423) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (1480) or a communication module (1490)). In one embodiment, the auxiliary processor (1423) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (1401) where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (1408)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0129] The memory (1430) can store various data used by at least one component (e.g., the processor (1420) or the sensor module (1476)) of the electronic device (1401). The data can include, for example, software (e.g., the program (1440)) and input data or output data for commands related thereto. The memory (1430) can include volatile memory (1432) or non-volatile memory (1434).

[0130] The program (1440) may be stored as software in memory (1430) and may include, for example, an operating system (1442), middleware (1444), or an application (1446).

[0131] The input module (1450) can receive commands or data to be used in a component of the electronic device (1401) (e.g., a processor (1420)) from an external source (e.g., a user) of the electronic device (1401). The input module (1450) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0132] The audio output module (1455) can output audio signals to the outside of the electronic device (1401). The audio output module (1455) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0133] The display module (1460) can visually provide information to an external party (e.g., a user) of the electronic device (1401). The display module (1460) may include, for example, a display, a holographic device, or a projector, and a control circuit for controlling the device. In one embodiment, the display module (1460) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0134] The audio module (1470) can convert sound into an electrical signal, or vice versa. According to one embodiment, the audio module (1470) can acquire sound through the input module (1450), output sound through the sound output module (1455), or an external electronic device (e.g., electronic device (1402)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (1401).

[0135] The sensor module (1476) can detect the operating status (e.g., power or temperature) of the electronic device (1401) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (1476) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0136] The interface (1477) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (1401) with an external electronic device (e.g., the electronic device (1402)). In one embodiment, the interface (1477) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0137] The connection terminal (1478) may include a connector through which the electronic device (1401) may be physically connected to an external electronic device (e.g., the electronic device (1402)). In one embodiment, the connection terminal (1478) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0138] The haptic module (1479) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. In one embodiment, the haptic module (1479) may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0139] The camera module (1480) can capture still images and videos. In one embodiment, the camera module (1480) may include one or more lenses, image sensors, image signal processors, or flashes.

[0140] The power management module (1488) can manage the power supplied to the electronic device (1401). According to one embodiment, the power management module (1488) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0141] A battery (1489) may power at least one component of the electronic device (1401). In one embodiment, the battery (1489) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0142] The communication module (1490) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (1401) and an external electronic device (e.g., electronic device (1402), electronic device (1404), or server (1408)), and the performance of communication through the established communication channel. The communication module (1490) may operate independently from the processor (1420) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (1490) may include a wireless communication module (1492) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (1494) (e.g., a local area network (LAN) communication module, or a power line communication module). Any of these communication modules may communicate with an external electronic device (1404) via a first network (1498) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (1499) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (1492) may use subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (1496) to verify or authenticate the electronic device (1401) within a communication network such as the first network (1498) or the second network (1499).

[0143] The wireless communication module (1492) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimizing terminal power and connecting multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency communications (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (1492) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (1492) may support various technologies for securing performance in high-frequency bands, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (1492) may support various requirements specified in the electronic device (1401), an external electronic device (e.g., the electronic device (1404)), or a network system (e.g., the second network (1499)). According to one embodiment, the wireless communication module (1492) may support a peak data rate (e.g., 20 Gbps or more) for eMBB implementation, a loss coverage (e.g., 164 dB or less) for mMTC implementation, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC implementation.

[0144] The antenna module (1497) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (1497) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (1497) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (1498) or the second network (1499), may be selected from the plurality of antennas by, for example, the communication module (1490). A signal or power may be transmitted or received between the communication module (1490) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (1497).

[0145] According to various embodiments, the antenna module (1497) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band.

[0146] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0147] According to one embodiment, commands or data may be transmitted or received between the electronic device (1401) and an external electronic device (1404) via a server (1408) connected to a second network (1499). Each of the external electronic devices (1402 or 1404) may be the same or a different type of device as the electronic device (1401). According to one embodiment, all or part of the operations executed in the electronic device (1401) may be executed in one or more of the external electronic devices (1402, 1404, or 1408). For example, when the electronic device (1401) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (1401) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (1401). The electronic device (1401) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (1401) may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (1404) may include an Internet of Things (IoT) device. The server (1408) may be an intelligent server utilizing machine learning and / or a neural network.According to one embodiment, an external electronic device (1404) or server (1408) may be included within the second network (1499). The electronic device (1401) may be applied to intelligent services (e.g., smart homes, smart cities, smart cars, or healthcare) based on 5G communication technology and IoT-related technology.

[0148] Figure 15a shows an example of a perspective view of a wearable device.

[0149] FIG. 15b illustrates an example of one or more hardware elements placed within a wearable device.

[0150] FIG. 15A illustrates an example of a perspective view of a wearable device. FIG. 15B illustrates an example of one or more hardware components arranged within the wearable device. According to one embodiment, a wearable device (1501) (e.g., the wearable device (1501) of FIG. 2 ) may have the form of glasses that can be worn on a body part (e.g., the head) of a user. The wearable device (1501) of FIGS. 15A and 15B may be an example of the electronic device (1401) of FIG. 14 . The wearable device (1501) may be referred to as the electronic device (1401). The wearable device (1501) may include a head-mounted display (HMD). For example, the housing of the wearable device (1501) may include a flexible material, such as rubber and / or silicone, that is configured to fit closely to a portion of the user's head (e.g., a portion of the face surrounding both eyes). For example, the housing of the wearable device (1501) may include one or more straps that can be twined around the user's head, and / or one or more temples that can be attached to an ear of the head.

[0151] Referring to FIG. 15A, according to one embodiment, a wearable device (1501) may include at least one display (1550) (e.g., display assembly (208) of FIG. 2) and a frame (1500) supporting at least one display (1550).

[0152] According to one embodiment, a wearable device (1501) can be worn on a part of a user's body. The wearable device (1501) can provide augmented reality (AR), virtual reality (VR), or mixed reality (MR) that combines augmented reality and virtual reality to a user wearing the wearable device (1501). For example, the wearable device (1501) can display a virtual reality image provided from at least one optical device (1582, 1584) of FIG. 15B on at least one display (1550) in response to a user's designated gesture acquired through the motion recognition cameras (1560-2, 1560-3) of FIG. 15B (e.g., one or more second cameras (210) of FIG. 2).

[0153] According to one embodiment, at least one display (1550) may provide visual information to a user. For example, at least one display (1550) may include a transparent or translucent lens. At least one display (1550) may include a first display (1550-1) and / or a second display (1550-2) spaced apart from the first display (1550-1). For example, the first display (1550-1) and the second display (1550-2) may be positioned at positions corresponding to the user's left and right eyes, respectively.

[0154] Referring to FIG. 15B, at least one display (1550) can provide visual information transmitted from external light to a user through a lens included in the at least one display (1550), and other visual information distinct from the visual information. The lens can be formed based on at least one of a Fresnel lens, a pancake lens, or a multi-channel lens. For example, at least one display (1550) can include a first surface (1531) and a second surface (1532) opposite to the first surface (1531). A display area can be formed on the second surface (1532) of the at least one display (1550). When a user wears the wearable device (1501), external light can be transmitted to the user by being incident on the first surface (1531) and transmitted through the second surface (1532). As another example, at least one display (1550) can display an augmented reality image combined with a virtual reality image provided from at least one optical device (1582, 1584) on a real screen transmitted through external light, in a display area formed on the second surface (1532).

[0155] In one embodiment, at least one display (1550) may include at least one waveguide (1533, 1534) that diffracts light emitted from at least one optical device (1582, 1584) and transmits the diffracted light to a user. The at least one waveguide (1533, 1534) may be formed based on at least one of glass, plastic, or polymer. A nano-pattern may be formed on at least a portion of the exterior or interior of the at least one waveguide (1533, 1534). The nano-pattern may be formed based on a grating structure having a polygonal and / or curved shape. Light incident on one end of the at least one waveguide (1533, 1534) may be propagated to the other end of the at least one waveguide (1533, 1534) by the nano-pattern. At least one waveguide (1533, 1534) may include at least one diffractive element (e.g., a diffractive optical element (DOE), a holographic optical element (HOE)), or at least one reflective element (e.g., a reflective mirror). For example, at least one waveguide (1533, 1534) may be arranged within the wearable device (1501) to guide a screen displayed by at least one display (1550) to the user's eyes. For example, the screen may be transmitted to the user's eyes based on total internal reflection (TIR) ​​occurring within the at least one waveguide (1533, 1534).

[0156] The wearable device (1501) can analyze an object included in a real image collected through a shooting camera (1560-4), combine a virtual object corresponding to an object to be provided with augmented reality among the analyzed objects, and display the virtual object on at least one display (1550). The virtual object can include at least one of text and an image regarding various information related to the object included in the real image. The wearable device (1501) can analyze the object based on a multi-camera such as a stereo camera. For the object analysis, the wearable device (1501) can perform spatial recognition (e.g., simultaneous localization and mapping (SLAM)) using the multi-camera and / or time-of-flight (ToF). A user wearing the wearable device (1501) can view an image displayed on at least one display (1550).

[0157] According to one embodiment, the frame (1500) may be configured as a physical structure that allows the wearable device (1501) to be worn on the user's body. According to one embodiment, the frame (1500) may be configured so that, when the user wears the wearable device (1501), the first display (1550-1) and the second display (1550-2) can be positioned corresponding to the user's left and right eyes. The frame (1500) may support at least one display (1550). For example, the frame (1500) may support the first display (1550-1) and the second display (1550-2) to be positioned corresponding to the user's left and right eyes.

[0158] Referring to FIG. 15A, the frame (1500) may include an area (1520) that at least partially contacts a portion of the user's body when the user wears the wearable device (1501). For example, the area (1520) of the frame (1500) that contacts a portion of the user's body may include an area that contacts a portion of the user's nose, a portion of the user's ear, and a portion of the side of the user's face that the wearable device (1501) comes into contact with. According to one embodiment, the frame (1500) may include a nose pad (1510) that contacts a portion of the user's body. When the wearable device (1501) is worn by the user, the nose pad (1510) may contact a portion of the user's nose. The frame (1500) may include a first temple (1504) and a second temple (1505) that contact another part of the user's body that is distinct from the part of the user's body.

[0159] For example, the frame (1500) may include a first rim (1501) that surrounds at least a portion of the first display (1550-1), a second rim (1502) that surrounds at least a portion of the second display (1550-2), a bridge (1503) that is disposed between the first rim (1501) and the second rim (1502), a first pad (1511) that is disposed along a portion of the edge of the first rim (1501) from one end of the bridge (1503), a second pad (1512) that is disposed along a portion of the edge of the second rim (1502) from the other end of the bridge (1503), a first temple (1504) that extends from the first rim (1501) and is fixed to a portion of the wearer's ear, and a second temple (1505) that extends from the second rim (1502) and is fixed to a portion of the ear opposite the ear. There are. The first pad (1511) and the second pad (1512) can be in contact with a part of the user's nose, and the first temple (1504) and the second temple (1505) can be in contact with a part of the user's face and a part of the user's ear. The temples (1504, 1505) can be rotatably connected to the rim through the hinge units (1506, 1507) of FIG. 15B. The first temple (1504) can be rotatably connected to the first rim (1501) through the first hinge unit (1506) disposed between the first rim (1501) and the first temple (1504). The second temple (1505) may be rotatably connected to the second rim (1502) via a second hinge unit (1507) disposed between the second rim (1502) and the second temple (1505). In one embodiment, the wearable device (1501) may use a touch sensor, a grip sensor, and / or a proximity sensor formed on at least a portion of a surface of the frame (1500) to identify an external object (e.g., a user's fingertip) touching the frame (1500) and / or a gesture performed by the external object.

[0160] According to one embodiment, the wearable device (1501) may include hardwares that perform various functions (e.g., the hardwares described above based on the block diagram of FIG. 2). For example, the hardwares may include a battery module (1570), an antenna module (1575), at least one optical device (1582, 1584), speakers (e.g., speakers 1555-1, 1555-2), a microphone (e.g., microphones 1565-1, 1565-2, 1565-3), a light-emitting module (not shown), and / or a printed circuit board (PCB) (1590) (e.g., a printed circuit board). The various hardwares may be arranged within the frame (1500).

[0161] According to one embodiment, microphones (e.g., microphones 1565-1, 1565-2, 1565-3) of a wearable device (1501) may be disposed on at least a portion of a frame (1500) to acquire sound signals. A first microphone (1565-1) disposed on a bridge (1503), a second microphone (1565-2) disposed on a second rim (1502), and a third microphone (1565-3) disposed on the first rim (1501) are illustrated in FIG. 15B , but the number and arrangement of the microphones (1565) are not limited to the embodiment of FIG. 15B . When the number of microphones (1565) included in the wearable device (1501) is two or more, the wearable device (1501) can identify the direction of a sound signal by using a plurality of microphones arranged on different parts of the frame (1500).

[0162] According to one embodiment, at least one optical device (1582, 1584) may project a virtual object onto at least one display (1550) to provide various image information to a user. For example, at least one optical device (1582, 1584) may be a projector. At least one optical device (1582, 1584) may be disposed adjacent to at least one display (1550) or may be included within at least one display (1550) as a part of at least one display (1550). According to one embodiment, the wearable device (1501) may include a first optical device (1582) corresponding to a first display (1550-1) and a second optical device (1584) corresponding to a second display (1550-2). For example, at least one optical device (1582, 1584) may include a first optical device (1582) disposed at an edge of a first display (1550-1) and a second optical device (1584) disposed at an edge of a second display (1550-2). The first optical device (1582) may transmit light to a first waveguide (1533) disposed on the first display (1550-1), and the second optical device (1584) may transmit light to a second waveguide (1534) disposed on the second display (1550-2).

[0163] In one embodiment, the camera (1560) may include a recording camera (1560-4), an eye tracking camera (ET CAM) (1560-1), and / or a motion recognition camera (1560-2, 1560-3). The recording camera (1560-4), the eye tracking camera (1560-1), and the motion recognition cameras (1560-2, 1560-3) may be positioned at different locations on the frame (1500) and may perform different functions. The eye tracking camera (1560-1) may output data indicating the position or gaze of the eyes of a user wearing the wearable device (1501). For example, the wearable device (1501) may detect the gaze from an image including the user's pupils obtained through the eye tracking camera (1560-1). The wearable device (1501) can identify an object (e.g., a real object and / or a virtual object) focused on by the user using the user's gaze acquired through the gaze tracking camera (1560-1). The wearable device (1501) that has identified the focused object can execute a function (e.g., gaze interaction) for interaction between the user and the focused object. The wearable device (1501) can express a part corresponding to the eye of an avatar representing the user in a virtual space using the user's gaze acquired through the gaze tracking camera (1560-1). The wearable device (1501) can render an image (or screen) displayed on at least one display (1550) based on the position of the user's eyes. For example, the visual quality (e.g., resolution, brightness, saturation, grayscale, PPI) of a first area related to the gaze within the image and the visual quality (e.g., resolution, brightness, saturation, grayscale, PPI) of a second area distinguished from the first area may be different from each other.The wearable device (1501) can obtain an image having a visual quality of a first region and a visual quality of a second region that match the user's gaze using foveated rendering. For example, if the wearable device (1501) supports an iris recognition function, user authentication can be performed based on iris information obtained using a gaze tracking camera (1560-1). An example in which the gaze tracking camera (1560-1) is positioned toward the user's right eye is illustrated in FIG. 15B, but the embodiment is not limited thereto, and the gaze tracking camera (1560-1) can be positioned solely toward the user's left eye, or toward both eyes.

[0164] In one embodiment, the capturing camera (1560-4) can capture an actual image or background to be aligned with a virtual image to implement augmented reality or mixed reality content. The capturing camera (1560-4) can be used to obtain a high-resolution image based on HR (high resolution) or PV (photo video). The capturing camera (1560-4) can capture an image of a specific object existing at a location viewed by the user and provide the image to at least one display (1550). The at least one display (1550) can display a single image in which information about an actual image or background including an image of the specific object obtained using the capturing camera (1560-4) and a virtual image provided through at least one optical device (1582, 1584) are superimposed. The wearable device (1501) can compensate for depth information (e.g., the distance between the wearable device (1501) and an external object acquired through a depth sensor) using an image acquired through the capture camera (1560-4). The wearable device (1501) can perform object recognition using an image acquired through the capture camera (1560-4). The wearable device (1501) can perform a function of focusing on an object (or subject) in an image (e.g., auto focus) and / or an optical image stabilization (OIS) function (e.g., anti-shake function) using the capture camera (1560-4). The wearable device (1501) can perform a pass-through function to display an image acquired through the capture camera (1560-4) by overlapping at least a portion of a screen representing a virtual space on at least one display (1550). In one embodiment, the capturing camera (1560-4) may be positioned on a bridge (1503) positioned between the first rim (1501) and the second rim (1502).

[0165] The gaze tracking camera (1560-1) can implement more realistic augmented reality by tracking the gaze of a user wearing a wearable device (1501) and matching the user's gaze with visual information provided to at least one display (1550). For example, when the wearable device (1501) looks straight ahead, the wearable device (1501) can naturally display environmental information related to the user's front at a location where the user is located on at least one display (1550). The gaze tracking camera (1560-1) can be configured to capture an image of the user's pupil to determine the user's gaze. For example, the gaze tracking camera (1560-1) can receive gaze detection light reflected from the user's pupil and track the user's gaze based on the position and movement of the received gaze detection light. In one embodiment, the gaze tracking camera (1560-1) can be positioned at positions corresponding to the user's left and right eyes. For example, the gaze tracking camera (1560-1) may be positioned within the first rim (1501) and / or the second rim (1502) to face the direction in which the user wearing the wearable device (1501) is positioned.

[0166] The gesture recognition camera (1560-2, 1560-3) can provide a specific event on a screen provided on at least one display (1550) by recognizing the movement of the user's entire body, such as the user's torso, hand, or face, or a part of the user's body. The gesture recognition camera (1560-2, 1560-3) can recognize the user's gesture (gesture recognition), obtain a signal corresponding to the gesture, and provide a display corresponding to the signal on at least one display (1550). The processor can identify the signal corresponding to the gesture, and perform a designated function based on the identification. The gesture recognition camera (1560-2, 1560-3) can be used to perform a spatial recognition function using SLAM and / or a depth map for 6 degrees of freedom pose (6 dof pose). The processor may perform gesture recognition and / or object tracking functions using the motion recognition cameras (1560-2, 1560-3). In one embodiment, the motion recognition cameras (1560-2, 1560-3) may be positioned on the first rim (1501) and / or the second rim (1502).

[0167] The camera (1560) included in the wearable device (1501) is not limited to the above-described gaze tracking camera (1560-1) and motion recognition cameras (1560-2, 1560-3). For example, the wearable device (1501) can identify an external object included in the FoV using a camera positioned toward the user's FoV. The wearable device (1501) can identify an external object based on a sensor for identifying the distance between the wearable device (1501) and the external object, such as a depth sensor and / or a time of flight (ToF) sensor. The camera (1560) positioned toward the FoV can support an autofocus function and / or an optical image stabilization (OIS) function. For example, the wearable device (1501) may include a camera (1560) (e.g., a face tracking (FT) camera) positioned toward the face to obtain an image including the face of a user wearing the wearable device (1501).

[0168] Although not shown, in one embodiment, the wearable device (1501) may further include a light source (e.g., an LED) that emits light toward a subject (e.g., a user's eyes, face, and / or an external object within the FoV) being captured using the camera (1560). The light source may include an infrared wavelength LED. The light source may be disposed on at least one of the frame (1500) and the hinge units (1506, 1507).

[0169] In one embodiment, the battery module (1570) may supply power to the electronic components of the wearable device (1501). In one embodiment, the battery module (1570) may be disposed within the first temple (1504) and / or the second temple (1505). For example, the battery module (1570) may be a plurality of battery modules (1570). The plurality of battery modules (1570) may be disposed within each of the first temple (1504) and the second temple (1505). In one embodiment, the battery module (1570) may be disposed at an end of the first temple (1504) and / or the second temple (1505).

[0170] The antenna module (1575) can transmit signals or power to the outside of the wearable device (1501), or receive signals or power from the outside. In one embodiment, the antenna module (1575) can be positioned within the first temple (1504) and / or the second temple (1505). For example, the antenna module (1575) can be positioned close to one surface of the first temple (1504) and / or the second temple (1505).

[0171] The speaker (1555) can output an audio signal to the outside of the wearable device (1501). The audio output module may be referred to as a speaker. In one embodiment, the speaker (1555) may be positioned within the first temple (1504) and / or the second temple (1505) so as to be positioned adjacent to the ear of a user wearing the wearable device (1501). For example, the speaker (1555) may include a second speaker (1555-2) positioned within the first temple (1504) and thus adjacent to the user's left ear, and a first speaker (1555-1) positioned within the second temple (1505) and thus adjacent to the user's right ear.

[0172] The light-emitting module (not shown) may include at least one light-emitting element. The light-emitting module may emit light of a color corresponding to a specific state or emit light with an action corresponding to a specific state to visually provide information regarding a specific state of the wearable device (1501) to the user. For example, when the wearable device (1501) requires charging, the wearable device (1501) may emit red light at a regular cycle. In one embodiment, the light-emitting module may be disposed on the first rim (1501) and / or the second rim (1502).

[0173] Referring to FIG. 15B, according to one embodiment, a wearable device (1501) may include a printed circuit board (PCB) (1590). The PCB (1590) may be included in at least one of the first temple (1504) or the second temple (1505). The PCB (1590) may include an interposer positioned between at least two sub-PCBs. One or more hardwares included in the wearable device (1501) (e.g., hardwares illustrated by different blocks in FIG. 4) may be positioned on the PCB (1590). The wearable device (1501) may include a flexible PCB (FPCB) for interconnecting the hardwares.

[0174] According to one embodiment, a wearable device (1501) may include at least one of a gyro sensor, a gravity sensor, and / or an acceleration sensor for detecting a posture of the wearable device (1501) and / or a posture of a body part (e.g., a head) of a user wearing the wearable device (1501). Each of the gravity sensor and the acceleration sensor may measure gravitational acceleration and / or acceleration based on mutually perpendicular designated three-dimensional axes (e.g., an x-axis, a y-axis, and a z-axis). The gyro sensor may measure an angular velocity of each of the designated three-dimensional axes (e.g., an x-axis, a y-axis, and a z-axis). At least one of the gravity sensor, the acceleration sensor, and the gyro sensor may be referred to as an inertial measurement unit (IMU). According to one embodiment, the wearable device (1501) may identify a user's motion and / or gesture performed to execute or terminate a specific function of the wearable device (1501) based on the IMU.

[0175] Figures 16a and 16b show an example of the appearance of a wearable device.

[0176] FIGS. 16A and 16B illustrate an example of an exterior appearance of a wearable device (1501) (e.g., the head-mounted electronic device (100) of FIG. 2 ). The wearable device (1501) of FIGS. 16A and 16B may be an example of the electronic device (1401) of FIG. 14 . According to one embodiment, an example of an exterior appearance of a first side (1610) of a housing of the wearable device (1501) is illustrated in FIG. 16A , and an example of an exterior appearance of a second side (1620) opposite to the first side (1610) is illustrated in FIG. 16B .

[0177] Referring to FIG. 16A, a first surface (1610) of a wearable device (1501) according to one embodiment may have a form attachable to a body part of a user (e.g., the face of the user). Although not shown, the wearable device (1501) may further include a strap for fixing to a body part of a user, and / or one or more temples (e.g., the first temple (1504) and / or the second temple (1505) of FIGS. 15A and 15B). A first display (1550-1) for outputting an image to a left eye among the user's two eyes, and a second display (1550-2) for outputting an image to a right eye among the two eyes, may be disposed on the first surface (1610). The wearable device (1501) may be formed on the first surface (1610) and may further include a rubber or silicone packing to prevent interference from light (e.g., ambient light) different from the light emitted from the first display (1550-1) and the second display (1550-2).

[0178] According to one embodiment, a wearable device (1501) may include cameras (1560-1) for photographing and / or tracking both eyes of a user adjacent to each of the first display (1550-1) and the second display (1550-2). The cameras (1560-1) may be referred to as the gaze tracking camera (1560-1) of FIG. 15B. According to one embodiment, a wearable device (1501) may include cameras (1560-5, 1560-6) for photographing and / or recognizing a face of a user. The cameras (1560-5, 1560-6) may be referred to as FT cameras. The wearable device (1501) can control an avatar representing the user in a virtual space based on the facial motion of the user identified using cameras (1560-5, 1560-6). For example, the wearable device (1501) can change the texture and / or shape of a part of the avatar (e.g., a part of the avatar representing a human face) using information obtained by cameras (1560-5, 1560-6) (e.g., an FT camera) and representing the facial expression of the user wearing the wearable device (1501).

[0179] Referring to FIG. 16B, a camera (e.g., cameras (1560-7, 1560-8, 1560-9, 1560-10, 1560-11, 1560-12)) and / or a sensor (e.g., a depth sensor (1630)) for obtaining information related to the external environment of the wearable device (1501) may be disposed on a second surface (1620) opposite to the first surface (1610) of FIG. 16A. For example, the cameras (1560-7, 1560-8, 1560-9, 1560-10) may be disposed on the second surface (1620) for recognizing external objects. Cameras (1560-7, 1560-8, 1560-9, 1560-10) may be referenced to the motion recognition cameras (1560-2, 1560-3) of FIG. 15b.

[0180] For example, using cameras (1560-11, 1560-12), the wearable device (1501) can obtain images and / or videos to be transmitted to each of the user's eyes. The camera (1560-11) can be positioned on the second face (1620) of the wearable device (1501) to obtain an image to be displayed through the second display (1550-2) corresponding to the right eye among the two eyes. The camera (1560-12) can be positioned on the second face (1620) of the wearable device (1501) to obtain an image to be displayed through the first display (1550-1) corresponding to the left eye among the two eyes. The cameras (1560-11, 1560-12) can be referred to as the shooting camera (1560-4) of FIG. 15B.

[0181] According to one embodiment, a wearable device (1501) may include a depth sensor (1630) disposed on a second face (1620) to identify a distance between the wearable device (1501) and an external object. Using the depth sensor (1630), the wearable device (1501) may obtain spatial information (e.g., a depth map) for at least a portion of the FoV of a user wearing the wearable device (1501). Although not shown, a microphone may be disposed on the second face (1620) of the wearable device (1501) to obtain sound output from an external object. The number of microphones may be one or more, depending on the embodiment.

[0182] Hereinafter, with reference to FIG. 17, the hardware or software configuration of the wearable device (1501) is described.

[0183] Fig. 17 illustrates an example of a block diagram of a wearable device. The wearable device (1501) of Fig. 17 (e.g., the head-mounted electronic device (100) of Fig. 2) may be an example of the electronic device (1501) of Fig. 14 or the wearable devices (1501) of Figs. 15A to 16B.

[0184] Referring to FIG. 17, a wearable device (100) according to one embodiment may include a processor (1710), a memory (1715), a display (1550) (e.g., the first display (1550-1) and / or the second display (1550-2) of FIGS. 15A, 15B, 16A, and 16B), and / or a sensor (1720). The processor (1710), the memory (1715), the display (1550), and / or the sensor (1720) may be electrically and / or operatively connected to each other by electronic components such as a communication bus (1702). In the present disclosure, the operative connection of the electronic components may include a direct connection established between the electronic components and / or an indirect connection established between the electronic components such that a first electronic component among the electronic components is controlled by a second electronic component among the electronic components. The type and / or number of electronic components included in the wearable device (100) is not limited to those illustrated in FIG. 17. For example, the wearable device (100) may include only some of the electronic components illustrated in FIG. 17.

[0185] A processor (1710) of a wearable device (100) according to one embodiment may include a circuit (e.g., a processing circuit) for processing data based on one or more instructions. The circuit for processing data may include, for example, an arithmetic and logic unit (ALU), a field programmable gate array (FPGA), a central processing unit (CPU), and / or an application processor (AP). In one embodiment, the wearable device (100) may include one or more processors. The processor (1710) may have a multi-core processor structure such as a dual core, a quad core, a hexa core, and / or an octa core. The multi-core processor structure of the processor (1710) may include a structure (e.g., a big-little structure) based on a plurality of core circuits that are distinguished by power consumption, clock, and / or calculation amount per unit time. In one embodiment comprising a processor (1710) having a multi-core processor architecture, the operations and / or functions of the present disclosure may be performed individually or collectively by one or more cores included in the processor (1710).

[0186] A memory (1715) of a wearable device (100) according to one embodiment may include electronic components for storing data and / or instructions input to and / or output from a processor (1710). The memory (1715) may include, for example, volatile memory such as random-access memory (RAM) and / or non-volatile memory such as read-only memory (ROM). The volatile memory may include, for example, at least one of dynamic RAM (DRAM), static RAM (SRAM), cache RAM, and pseudo SRAM (PSRAM). The non-volatile memory may include, for example, at least one of programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, hard disk, compact disc, and embedded multi media card (eMMC). In one embodiment, memory (1715) may be referred to as storage.

[0187] In one embodiment, a display (1550) of a wearable device (100) can output visualized information to a user of the wearable device (100). The display (1550), which is arranged in front of the eyes of a user wearing the wearable device (100), can be arranged on at least a portion of a housing of the wearable device (100) (e.g., the first display (1550-1) and / or the second display (1550-2) of FIGS. 15A, 15B, 16A, and 16B). For example, the display (1550) can be controlled by a processor (1710) including circuits such as a CPU, a GPU (graphics processing unit), and / or a DPU (display processing unit), to output visualized information to the user. The display (1550) can include a flexible display, a flat panel display (FPD), and / or electronic paper. The display (1550) may include a liquid crystal display (LCD), a plasma display panel (PDP), and / or one or more light emitting diodes (LEDs). The LEDs may include organic LEDs (OLEDs). The embodiment is not limited thereto, and for example, if the wearable device (100) includes a lens for transmitting external light (or ambient light), the display (1550) may include a projector (or projection assembly) for projecting light onto the lens. In one embodiment, the display (1550) may be referred to as a display panel and / or a display module. The pixels included in the display (1550) may be arranged to face either of the user's eyes when the wearable device (100) is worn by the user.For example, the display (1550) may include display areas (or active areas) corresponding to each of the user's two eyes.

[0188] In one embodiment, the sensor (1720) of the wearable device (100) may generate electrical information that may be processed by the processor (1710) and / or the memory (1715) from non-electronic information related to the wearable device (100). For example, the sensor (1720) may include a global positioning system (GPS) sensor for detecting the geographic location of the wearable device (100). In addition to the GPS method, the sensor (1720) may generate information indicating the geographic location of the wearable device (100) based on a global navigation satellite system (GNSS) such as, for example, Galileo or Beidou (compass). The above information may be stored in memory (1715), processed by a processor (1710), and / or transmitted to another electronic device distinct from the wearable device (100) via a communication circuit.

[0189] According to one embodiment, one or more instructions (or commands) representing data to be processed, calculations to be performed, and / or operations to be performed by the processor (1710) of the wearable device (100) may be stored in the memory (1715) of the wearable device (100). A set of one or more instructions may be referred to as a program, firmware, an operating system, a process, a routine, a sub-routine, and / or a software application (hereinafter, “application”). For example, the wearable device (100) and / or the processor (1710) may perform at least one of the operations of FIGS. 3, 6, and 9 when a set of a plurality of instructions distributed in the form of an operating system, firmware, a driver, a program, and / or a software application is executed. Hereinafter, the fact that a software application is installed in a wearable device (100) may mean that one or more instructions provided in the form of a software application (or package) are stored in a memory (1715), and that the one or more applications are stored in a format executable by the processor (1710) (e.g., a file having an extension specified by the operating system of the wearable device (100)). As an example, the application may include a program and / or a library related to a service provided to a user.

[0190] Referring to FIG. 17, programs installed in the wearable device (100) may be included in any one of different layers, including an application layer (1740), a framework layer (1750), and / or a hardware abstraction layer (HAL) (1780), based on the target. For example, programs (e.g., modules or drivers) designed to target the hardware (e.g., the display (1550), and / or the sensor (1720)) of the wearable device (100) may be included in the hardware abstraction layer (1780). The framework layer (1750) may be referred to as an XR framework layer from the perspective of including one or more programs for providing an XR (extended reality) service. For example, the layers illustrated in FIG. 17 may be logically (or for convenience of explanation) separated, and may not mean that the address space of the memory (1715) is separated by the layers.

[0191] For example, within the framework layer (1750), programs designed to target at least one of the hardware abstraction layer (1780) and / or the application layer (1740) (e.g., a position tracker (1771), a space recognizer (1772), a gesture tracker (1773), an eye-gaze tracker (1774), and / or a face tracker (1775)) may be included. The programs included in the framework layer (1750) may provide an application programming interface (API) that is executable (or callable) based on other programs.

[0192] For example, a program designed to target users of a wearable device (100) may be included within the application layer (1740). As an example of programs included in the application layer (1740), an extended reality (XR) system user interface (UI) (1741) and / or an XR application (1742) are exemplified, but the embodiment is not limited thereto. For example, programs (e.g., software applications) included in the application layer (1740) may call an API to cause execution of functions supported by programs included in the framework layer (1750).

[0193] For example, the wearable device (100) may display one or more visual objects on the display (1550) for performing interaction with the user based on the execution of the XR system UI (1741). A visual object may refer to an object that can be placed within a screen for transmitting and / or interacting with information, such as text, an image, an icon, a video, a button, a checkbox, a radio button, a text box, a slider, and / or a table. A visual object may be referred to as a visual guide, a virtual object, a visual element, a UI element, a view object, and / or a view element. The wearable device (100) may provide the user with functions available within a virtual space based on the execution of the XR system UI (1741).

[0194] Referring to FIG. 17, a lightweight renderer (1743) and / or an XR plug-in (1744) is illustrated to be included within the XR system UI (1741), but is not limited thereto. For example, based on the XR system UI (1741), the processor (1710) may execute a lightweight renderer (1743) and / or an XR plug-in (1744) within the framework layer (1750).

[0195] For example, the wearable device (100) may acquire resources (e.g., APIs, system processes, and / or libraries) used to define, create, and / or execute a rendering pipeline that allows partial changes based on the execution of a lightweight renderer (1743). The lightweight renderer (1743) may be referred to as a lightweight render pipeline in terms of defining a rendering pipeline that allows partial changes. The lightweight renderer (1743) may include a renderer built prior to the execution of a software application (e.g., a prebuilt renderer). For example, the wearable device (100) may acquire resources (e.g., APIs, system processes, and / or libraries) used to define, create, and / or execute an entire rendering pipeline based on the execution of an XR plug-in (1744). The XR plugin (1744) can be referred to as an open XR native client from the perspective of defining (or configuring) the entire rendering pipeline.

[0196] For example, the wearable device (100) may display a screen representing at least a portion of a virtual space on the display (1550) based on the execution of the XR application (1742). The XR plug-in (1744-1) included in the XR application (1742) may include instructions that support functions similar to those of the XR plug-in (1744) of the XR system UI (1741). Descriptions of the XR plug-in (1744-1) that overlap with those of the XR plug-in (1744) may be omitted. The wearable device (100) may cause the execution of the virtual space manager (1751) based on the execution of the XR application (1742).

[0197] For example, the wearable device (100) may display an image on the display (1550) in a virtual space based on the execution of the application (1745). The application (1745) may be configured to output image information for displaying a two-dimensional image. The wearable device (100) may cause the execution of the virtual space manager (1751) based on the execution of the application (1745). The wearable device (100) may generate dual image information to display the two-dimensional image in a three-dimensional virtual space based on the execution of the application (1745). Here, the dual image information may include first image information for the left eye and second image information for the right eye in consideration of binocular parallax. In order to display the two-dimensional image in the three-dimensional virtual space, the wearable device (100) may generate the dual image information based on the image information for displaying the two-dimensional image.

[0198] According to one embodiment, the wearable device (100) may provide a virtual space service based on the execution of the virtual space manager (1751). For example, the virtual space manager (1751) may include a platform for supporting the virtual space service. Based on the execution of the virtual space manager (1751), the wearable device (100) may identify a virtual space formed based on the user's location indicated by data acquired through the sensor (1730), and may display at least a portion of the virtual space on the display (1550). The virtual space manager (1751) may be referred to as a composition presentation manager (CPM).

[0199] For example, the virtual space manager (1751) may include a runtime service (1752). As an example, the runtime service (1752) may be referred to as an OpenXR runtime module (or an OpenXR runtime program). The wearable device (100) may execute at least one of a user's pose prediction function, a frame timing function, and / or a spatial input function based on the execution of the runtime service (1752). As an example, the wearable device (100) may perform rendering for a virtual space service for the user based on the execution of the runtime service (1752). For example, a function related to a virtual space, executable by the application layer (1740), may be supported based on the execution of the runtime service (1752).

[0200] For example, the virtual space manager (1751) may include a pass-through manager (1753). Based on the execution of the pass-through manager (1753), the wearable device (100) may display an image and / or video representing an actual space acquired through an external camera on at least a portion of the screen while displaying a screen representing a virtual space on the display (1550).

[0201] For example, the virtual space manager (1751) may include an input manager (1754). The wearable device (100) may identify data (e.g., sensor data) obtained by executing one or more programs included in the recognition service layer (1770) based on the execution of the input manager (1754). The wearable device (100) may use the obtained data to identify a user input related to the wearable device (100). The user input may be related to a motion (e.g., a hand gesture), gaze, and / or speech of the user identified by a sensor (1720) (e.g., an image sensor (1721) such as an external camera). The user input may be identified based on an external electronic device connected (or paired) via a communication circuit.

[0202] For example, the perception abstract layer (1760) can be used for data exchange between the virtual space manager (1751) and the perception service layer (1770). From the perspective of being used for data exchange between the virtual space manager (1751) and the perception service layer (1770), the perception abstract layer (1760) can be referred to as an interface. For example, the perception abstract layer (1760) can be referenced as OpenPX. The perception abstract layer (1760) can be used for a perception client and a perception service.

[0203] According to one embodiment, the recognition service layer (1770) may include one or more programs for processing data acquired from the sensor (1720). The one or more programs may include at least one of a position tracker (1771), a space recognizer (1772), a gesture tracker (1773), and / or an eye tracker (1774). The type and / or number of the one or more programs included in the recognition service layer (1770) are not limited to those illustrated in FIG. 17.

[0204] For example, the wearable device (100) can identify the pose of the wearable device (100) using the sensor (1730) based on the execution of the position tracker (1771). The wearable device (100) can identify the 6 degrees of freedom pose (6 dof pose) of the wearable device (100) using data acquired using an external camera (e.g., an image sensor (1721)) and / or an IMU (e.g., a motion sensor (1722) including a gyro sensor, an acceleration sensor, and / or a geomagnetic sensor) based on the execution of the position tracker (1771). The position tracker (1771) may be referred to as a head tracking (HeT) module (or head tracker, head tracking program).

[0205] For example, the wearable device (100) may obtain information for providing a three-dimensional virtual space corresponding to the surrounding environment (e.g., external space) of the wearable device (100) (or the user of the wearable device (100)) based on the execution of the space recognizer (1772). The wearable device (100) may reproduce the surrounding environment of the wearable device (100) in three dimensions using data obtained using an external camera (e.g., an image sensor (1721)) based on the execution of the space recognizer (1772). The wearable device (100) may identify at least one of a plane, a slope, and stairs based on the surrounding environment of the wearable device (100) reproduced in three dimensions based on the execution of the space recognizer (1772). The space recognizer (1772) may be referred to as a scene understanding (SU) module (or a scene recognition program).

[0206] For example, the wearable device (100) may identify (or recognize) a pose and / or gesture of a hand of a user of the wearable device (100) based on the execution of the gesture tracker (1773). As an example, the wearable device (100) may identify a pose and / or gesture of a hand of a user using data acquired from an external camera (e.g., an image sensor (1721)) based on the execution of the gesture tracker (1773). As an example, the wearable device (100) may identify a pose and / or gesture of a hand of a user based on data (or images) acquired using an external camera based on the execution of the gesture tracker (1773). The gesture tracker (1773) may be referred to as a hand tracking (HaT) module (or hand tracking program) and / or a gesture tracking module.

[0207] For example, the wearable device (100) may identify (or track) eye movements of a user of the wearable device (100) based on the execution of the gaze tracker (1774). As an example, the wearable device (100) may identify eye movements of the user using data acquired from a gaze tracking camera (e.g., an image sensor (1721)) based on the execution of the gaze tracker (1774). The gaze tracker (1774) may be referred to as an eye tracking (ET) module (or eye tracking program) and / or a gaze tracking module.

[0208] For example, the recognition service layer (1770) of the wearable device (100) may further include a face tracker (1775) for tracking the user's face. For example, the wearable device (100) may identify (or track) the movement of the user's face and / or the user's expression based on the execution of the face tracker (1775). The wearable device (100) may estimate the user's expression based on the movement of the user's face based on the execution of the face tracker (1775). As an example, the wearable device (100) may identify the movement of the user's face and / or the user's expression based on data (e.g., images and / or videos) acquired using a camera (1725) (e.g., a camera facing at least a portion of the user's face) based on the execution of the face tracker (1775).

[0209] Referring to FIG. 17, the renderer (1790) may include instructions for rendering images in a three-dimensional virtual space. The processor (1710) executing the renderer (1790) may obtain at least one image to be at least partially displayed in the display area of ​​the display (1550) in a software application. For example, the processor (1710) executing the renderer (1790) may determine the location of the area in which an application (e.g., XR application (1742), application (1745)) is to be rendered. The processor (1710) executing the renderer (1790) may generate an image of the application to be displayed on the display (1550). The renderer (1790) may synthesize images to generate a composite image to be displayed on the display (1550).

[0210] For example, the processor (1710) executing the renderer (1790) can divide the display area of ​​the display (1550) into a foveated portion (or may be referred to as the foveated area) and a peripheral portion (or may be referred to as the residual area) using the gaze position calculated using the position tracker (1771) and / or the gaze tracker (1774). For example, the processor (1710) detecting the coordinate values ​​of the gaze position can determine the portion of the display area including the coordinate values ​​as the foveated area. The DPU executing the renderer (1790) can obtain at least one image corresponding to each of the foveated area and the residual area, and having a size smaller than the size of the entire display area of ​​the display (1550) or a resolution smaller than the resolution of the display area.

[0211] The processor (1710) executing the renderer (1790) may obtain or generate a composite image to be displayed on the display (1550) by synthesizing an image corresponding to the foveated area and an image corresponding to the surrounding area. For example, the processor (1710) may perform upscaling to enlarge the image corresponding to the surrounding area to the size of the entire display area of ​​the display (1550). On the enlarged image, the processor (1710) may combine the image corresponding to the foveated area to generate a composite image to be displayed on the display (1550). Along the boundary line of the image corresponding to the foveated area, the processor (1710) may apply a visual effect, such as blur, to blend the enlarged image and the image corresponding to the foveated area.

[0212] Fig. 18 shows an example of a block diagram of an electronic device for displaying an image in a virtual space.

[0213] Fig. 18 illustrates an example of a block diagram of an electronic device (1501) (e.g., a wearable device (100)) for displaying an image in a virtual space. In Fig. 18, an example of executing multiple programs / instructions for displaying an image in a virtual space is described. The multiple programs / instructions may all be executed in one processor (e.g., an AP) or may be executed by multiple processors (e.g., an AP, a GPU (graphics processing unit), an NPU (neural processing unit)). The meaning of being executable by the multiple processors means that some programs / instructions may be executed by a first processor and other some programs / instructions may be executed by a second processor different from the first processor.

[0214] Referring to FIG. 18, the electronic device (1501) may execute a virtual space manager (1850) (e.g., the virtual space manager (1751) of FIG. 17, CPM) to render an image in a virtual space. For the virtual space manager (1850), at least some of the descriptions of the virtual space manager (1751) of FIG. 17 may be referenced. The virtual space manager (1850) may include a platform for supporting a virtual space service. The virtual space manager (1850) may include a runtime service (1851) (e.g., OpenXR Runtime), a panel renderer (1852) (e.g., 2D Panel Render), and an XR compositor (1853). The electronic device (1501) may execute at least one of a user's pose prediction function, a frame timing function, and / or a spatial input function based on the execution of the runtime service (1851). For the runtime service (1851), at least some of the descriptions of the runtime service (1752) of FIG. 17 may be referred to. The electronic device (1501) may display at least one image (video) on a panel (e.g., a 2D panel) to implement a virtual space through the display based on the execution of the panel rendering (1852). For example, the electronic device (1501) may display a rendering image corresponding to RGB information (1866) for the panel from the spatialization manager (1840) described below through the display (e.g., the display (1550)). The electronic device (1501) may synthesize an image of an actual area captured by a camera in the virtual space (hereinafter, a pass-through image) with a virtual area image based on the execution of the XR compositor (1853). For example, the electronic device (1501) can generate a composite image by merging the pass-through image and the virtual area image based on the execution of the XR synthesis unit (1853).The electronic device (1501) may transmit the generated composite image to a display buffer so that the composite image is displayed. The electronic device (1501) may identify a virtual space through a virtual space manager (1850) and display at least a portion of the virtual space on the display (1550). The virtual space manager (1850) may be referred to as a CPM. The electronic device (1501) may execute the virtual space manager (1850) to render an image corresponding to at least a portion of the virtual space.

[0215] According to one embodiment, the electronic device (1501) may execute a spatialization manager (1840). The spatialization manager (1840) may perform processes for displaying an image in a three-dimensional virtual space. The electronic device (1501) may perform preprocessing based on the execution of the spatialization manager (1840) so that the image can be rendered in the three-dimensional virtual space through the virtual space manager (1850). For example, the electronic device (1501) may perform at least some of the functions of the renderer (1790) of FIG. 17 based on the execution of the spatialization manager (1840). The electronic device (1501) may process image information provided by an application (e.g., an XR application (1810), an application (1820) that provides a general 2D screen other than XR, and an application that provides a system UI (1830)) based on the execution of the spatialization manager (1840). A spatialization manager (1840) (e.g., Space Flinger) may include a system scene manager (1841) (e.g., System scene), an input manager (1842) (e.g., Input Routing), and a lightweight rendering engine (1843) (e.g., Impress Engine). The system scene manager (1841) may be executed to display a system UI (1830). System UI-related information (1864) may be transmitted to the system scene manager (1841) from a program (e.g., API) that provides the system UI (1830). The system UI-related information (1864) may be obtained through a spatializer API and / or a same-process private API. The spatialization manager (1840) may determine the layout (e.g., location, display order) of the screen of the system UI (1830) in a three-dimensional space through pre-allocated resources.The system screen manager (1841) may transmit image information (1867) for rendering the screen of the system UI (1830) to the virtual space manager (1850) according to the layout. The input manager (1842) may be configured to process user input (e.g., user input on a system screen or an app screen). The impression engine (1843) may be a renderer for image generation (e.g., a lightweight renderer (1743)). For example, the impression engine (1843) may be used to display the system UI (1830). According to one embodiment, the spatialization manager (1840) may include a lightweight rendering engine (1843) for rendering the system UI. According to one embodiment, when the lightweight rendering engine (1843) does not have sufficient resources to render an avatar used in the HMD, at least one external rendering engine may be used. At this time, to resolve compatibility issues with external rendering (e.g., 3rd party engines), an external rendering engine support module may be added within the spatialization manager (1840).

[0216] According to one embodiment, the electronic device can execute an application. For example, in response to the execution of an XR application (1810) (e.g., an XR application (1742), a 3D game, an XR map, or other immersive application), the electronic device can execute a virtual space manager (1850). The electronic device (1501) can provide dual image information (1861) provided from the XR application (1810) to the virtual space manager (1850). In order to display an image in a three-dimensional space, the dual image information (1861) can include two pieces of image information that take binocular parallax into account. For example, the dual image information (1861) can include first image information for the user's left eye and second image information for the user's right eye for rendering in a three-dimensional virtual space. Hereinafter, in the present disclosure, the term dual image information is used to refer to image information for displaying images for both eyes in a three-dimensional space. In addition to the dual image information, the above dual image information may also include binocular image information, dual image information, dual image data, dual images, binocular image data, stereoscopic image information, 3D image information, spatial image information, spatial image data, 15D-3D conversion data, dimensional conversion image data, binocular parallax image data, and / or equivalent technical terms. The electronic device (1501) can generate a composite image by merging image layers through the virtual space manager (1850). The electronic device (1501) can transmit the generated composite image to a display buffer. The composite image can be displayed on the display (1550) of the electronic device (1501).

[0217] According to one embodiment, the electronic device can execute at least one application among an XR application (1810) and other applications (1820) (e.g., a first application (1820-1), a second application (1820-2), ..., an Nth application (1820-N)). According to one embodiment, the application (1820) can be configured to output image information for displaying a two-dimensional image. In other words, the application (1820) can provide a two-dimensional image. For example, the application (1820) can be a video application, a schedule application, or an application (1820) can be an Internet browser application. If it is assumed that in response to the execution of the application (1820), image information (1862) provided from the application (1820) is provided to the virtual space manager (1850). Since the image information (1862) only has x-coordinates and y-coordinates within a two-dimensional plane, it may be difficult to consider the chronological relationship (i.e., the distance from the user) between other applications centered on the user. Even when displaying an application (1820) that provides a general 2D screen, the electronic device (1501) may execute the spatialization manager (1840) to provide dual image information to the virtual space manager (1850). For example, based on the execution of the spatialization manager (1840), the electronic device (1501) may receive application-related information (1863) from the first application (1820-1). For example, the application-related information (1863) may include image information representing a two-dimensional image of the first application (1820-1) (e.g., information including RGB per pixel) and / or content information in the first application (1820-1) (e.g., characteristics of content executed in the first application, type of content). Application related information (1863) can be obtained via the spatializer API.Based on the execution of the spatialization manager (1840), the electronic device (1501) can identify information about the location of the area to be rendered by the first application (1820-1) and the size of the area to be rendered (hereinafter, location information). Based on the execution of the spatialization manager (1840), the electronic device (1501) can generate dual image information (1865, e.g., RGBx2) that takes into account the user's binocular disparity through the image information and the location information. Based on the execution of the spatialization manager (1840), the electronic device (1501) can provide the dual image information (1865) to the virtual space manager (1850). By converting a simple two-dimensional image into the dual image information (1865), a problem that occurs when the image information (1862) is directly transmitted to the virtual space manager (1850) can be resolved. Additionally, since at least some of the functions for displaying images in a virtual space are performed by the spatialization manager (1840) instead of the virtual space manager (1850), the burden on the virtual space manager (1850) can be reduced.

[0218] The head-wearable electronic device (e.g., the head-wearable electronic device (100)) as described above may include a memory (e.g., the memory (206)) for storing instructions. The head-wearable electronic device (e.g., the head-wearable electronic device (100)) may include a communication circuit (e.g., the communication circuit (205)). The head-wearable electronic device (e.g., the head-wearable electronic device (100)) may include one or more cameras (e.g., one or more first cameras (209)) configured to acquire images of an eye of a user (e.g., the user (120)) wearing the head-wearable electronic device (e.g., the head-wearable electronic device (100)). The head-wearable electronic device (e.g., the head-wearable electronic device (100)) may include a display assembly (e.g., the display assembly (208)) including at least one display. The head-wearable electronic device (e.g., the head-wearable electronic device (100)) may include at least one processor. The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device (e.g., the head-wearable electronic device (100)) to display, through the display assembly (e.g., the display assembly (208)), UI objects (e.g., first UI objects (130)) overlaid on an environment (e.g., environment (150)) surrounding the head-wearable electronic device (e.g., the head-wearable electronic device (100)) shown through the display assembly (e.g., the display assembly (208)).The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device (e.g., the head-wearable electronic device (100)) to detect, using images acquired through the one or more cameras (e.g., the one or more first cameras (209)), a gaze directed toward another wearable device (e.g., another wearable device (110)) positioned within the environment (e.g., the environment (150)) and worn by a user (e.g., the user (120)) of the head-wearable electronic device (e.g., the head-wearable electronic device (100)) while displaying the UI object first UI objects (e.g., the first UI objects (130)) overlaid on the environment (e.g., the environment (150)), as viewed through the display assembly (e.g., the display assembly (208)). The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device (e.g., the head-wearable electronic device (100)) to display, on the environment (e.g., the environment (150)), a user interface (e.g., the user interface (490)), including UI object second UI objects (e.g., the second UI objects (430)), arranged in conjunction with the other wearable device. The UI object second UI objects (e.g., the second UI objects (430)) may include UI objects that respectively provide functions provided by the UI object first UI objects (e.g., the first UI objects (130)). The user interface may be controlled according to a user input received with respect to the other wearable device.

[0219] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to receive, via the communication circuitry, from the other wearable device, information about the user input received for the other wearable device while displaying the user interface arranged in conjunction with the other wearable device. The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to recognize the user input received for the other wearable device as user input received for the user interface.

[0220] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to detect, using images acquired through the one or more cameras, a gaze directed toward the user interface while displaying the user interface arranged in conjunction with the other wearable device. The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to receive, through the communication circuitry, information about the user input received for the other wearable device from the other wearable device after detecting the gaze while displaying the user interface arranged in conjunction with the other wearable device. The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to, in response to the receiving, apply the user input received for the other wearable device, and the other user input received for the head-wearable electronic device, as the user input received for the user interface.

[0221] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to move a location of one of the second UI objects from a first location to a second location in response to the user input.

[0222] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to execute a software application corresponding to one of the second UI objects in response to the user input.

[0223] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to, in response to the user input, cease displaying at least one of the second UI objects via the display assembly. The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to, in response to the user input, display a third UI object via the display assembly, the third UI object being distinct from each of the second UI objects.

[0224] In one embodiment, the one or more cameras may be one or more first cameras (e.g., one or more first cameras (209)). The images may be first images. The head-wearable electronic device may further include one or more second cameras (e.g., one or more second cameras (210)) configured to acquire images of an environment surrounding the head-wearable electronic device. The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to identify a gesture associated with the other wearable device using second images acquired through the one or more second cameras (e.g., one or more second cameras (210)). The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to control the user interface based on the identification.

[0225] According to one embodiment, each of the second UI objects may correspond to each of the first UI objects.

[0226] According to one embodiment, the user interface may include a number of second UI objects that are greater than the number of first UI objects displayed through the display assembly, overlapping the environment surrounding the head-wearable electronic device shown through the display assembly.

[0227] In one embodiment, the size of each of the second UI objects may be smaller than the size of each of the first UI objects. The user interface may be in a form in which the second UI objects included in the user interface surround the other wearable device.

[0228] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to display, through the display assembly, a visual affordance (e.g., visual affordance (730)) corresponding to the other wearable device, as an overlay on the environment surrounding the head-wearable electronic device as seen through the display assembly. The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to detect, using images acquired through the one or more cameras, a gaze directed toward the visual affordance (e.g., visual affordance (730)) displayed through the display assembly. The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to transmit a signal to the other wearable device for activating a mode in which the user interface is controlled in response to the user input received for the other wearable device through the communication circuit based on the detection.

[0229] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to detect, while the mode is activated, a gaze directed toward another visual affordance (e.g., visual affordance (830)) displayed through the display assembly using images acquired through the one or more cameras. The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to transmit, based on the detection, another signal to the other wearable device via the communication circuitry to deactivate the mode in which the user interface is controlled in accordance with the user input received for the other wearable device.

[0230] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to receive a signal from the other wearable device to activate a mode in which the user interface is controlled in response to the user input received for the other wearable device via the communication circuit.

[0231] In one embodiment, the one or more cameras may be one or more first cameras. The images may be first images. The head-wearable electronic device may further include one or more second cameras configured to acquire images of an environment surrounding the head-wearable electronic device. The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to identify a gesture associated with the other wearable device using the second images acquired through the one or more second cameras. The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to transmit a signal to the other wearable device for activating a mode in which the user interface is controlled in response to the user input received for the other wearable device through the communication circuitry based on the identification.

[0232] In one embodiment, the one or more cameras may be one or more first cameras. The images may be first images. The head-wearable electronic device may further include one or more second cameras configured to acquire images of an environment surrounding the head-wearable electronic device. The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to identify a quick response (QR) code (e.g., QR code 740) using the second images acquired through the one or more second cameras. The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to transmit a signal to the other wearable device for activating a mode in which the user interface is controlled according to the user input received for the other wearable device through the communication circuitry based on the identification.

[0233] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to receive information including the user input from the other wearable device via the communication circuit. The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to provide a haptic notification in response to the receiving.

[0234] A method performed by a head-wearable electronic device having a communication circuit (e.g., communication circuit (205)), one or more cameras (e.g., one or more first cameras (209)) configured to acquire images of an eye of a user wearing the head-wearable electronic device, and a display assembly (e.g., display assembly (208)) including at least one display, may include an operation of displaying, through the display assembly (e.g., display assembly (208)), UI object first UI objects (e.g., first UI objects (130)) as an overlay on an environment (e.g., environment (150)) surrounding the head-wearable electronic device as viewed through the display assembly (e.g., display assembly (208)). The method may include an operation of detecting a gaze directed toward another wearable device (e.g., another wearable device (110)) positioned within the environment (e.g., environment (150)) and worn by a user (e.g., user (120)) of the head-wearable electronic device, using images acquired through the one or more cameras (e.g., one or more first cameras (209)) while displaying the UI object first UI objects (e.g., first UI objects (130)) overlaid on the environment (e.g., environment (150)) and viewed through the display assembly (e.g., display assembly (208)). The method may include, based on the detection, an operation of displaying a user interface (e.g., a user interface (490)) including second UI objects (e.g., second UI objects (430)) and arranged in conjunction with the other wearable device, on the environment (e.g., the environment (150)), through the display assembly (e.g., the display assembly (208)).The above UI object second UI objects (e.g., second UI objects (430)) may include UI objects that each provide functions provided by the UI object first UI objects (e.g., first UI objects (130)). The UI object UI object The user interface may be controlled according to user input received for the other wearable device.

[0235] In one embodiment, the method may include an operation of receiving, from the other wearable device, information about the user input received for the other wearable device through the communication circuit while displaying the user interface arranged in conjunction with the other wearable device. The method may include an operation of recognizing the user input received for the other wearable device as a user input received for the user interface.

[0236] In one embodiment, the head-wearable electronic device may further include a communication circuit. The method may include an operation of detecting a gaze toward the user interface using images acquired through the one or more cameras while displaying the user interface arranged in conjunction with the other wearable device. The method may include an operation of receiving, from the other wearable device through the communication circuit, information about the user input received for the other wearable device after detecting the gaze while displaying the user interface arranged in conjunction with the other wearable device. The method may include an operation of applying, in response to the reception, the user input received for the other wearable device among the user input received for the other wearable device and the other user input received for the head-wearable electronic device, the user input received for the other wearable device as a user input received for the user interface.

[0237] In one embodiment, the method may include, in response to the user input, moving a location of one of the second UI objects from a first location to a second location.

[0238] According to one embodiment, the method may include, in response to the user input, executing a software application corresponding to one of the second UI objects.

[0239] In one embodiment, the method may include, in response to the user input, an operation of stopping displaying at least one of the second UI objects through the display assembly. The method may include, in response to the user input, an operation of displaying a third UI object, distinct from each of the second UI objects, through the display assembly.

[0240] In one embodiment, the one or more cameras may be one or more first cameras. The images may be first images. The method may further include one or more second cameras (e.g., one or more second cameras (210)) configured to acquire images of an environment surrounding the head-wearable electronic device. The method may include an operation of identifying a gesture associated with the other wearable device using second images acquired through the one or more second cameras (e.g., one or more second cameras (210)). The method may include an operation of controlling the user interface based on the identification.

[0241] According to one embodiment, each of the second UI objects may correspond to each of the first UI objects.

[0242] According to one embodiment, the user interface may include a number of second UI objects that are greater than the number of first UI objects displayed through the display assembly, overlapping the environment surrounding the head-wearable electronic device shown through the display assembly.

[0243] In one embodiment, the size of each of the second UI objects may be smaller than the size of each of the first UI objects. The user interface may be in a form in which the second UI objects included in the user interface surround the other wearable device.

[0244] In one embodiment, the method may include an operation of displaying, through the display assembly, a visual affordance (e.g., a visual affordance (730)) corresponding to the other wearable device, as an overlay on the environment surrounding the head-wearable electronic device shown through the display assembly. The method may include an operation of detecting, using images acquired through the one or more cameras, a gaze directed toward the visual affordance (e.g., a visual affordance (730)) displayed through the display assembly. The method may include an operation of transmitting, based on the detection, a signal to the other wearable device for activating a mode in which the user interface is controlled according to the user input received for the other wearable device through the communication circuit.

[0245] In one embodiment, the method may include detecting, while the mode is activated, a gaze directed toward another visual affordance (e.g., another visual affordance (830)) displayed through the display assembly using images acquired through the one or more cameras. The method may include transmitting, based on the detection, another signal to the other wearable device through the communication circuitry for deactivating the mode in which the user interface is controlled according to the user input received for the other wearable device.

[0246] In one embodiment, the method may include receiving a signal from the other wearable device to activate a mode in which the user interface is controlled according to the user input received for the other wearable device through the communication circuit.

[0247] In one embodiment, the one or more cameras may be one or more first cameras. The images may be first images. The method may further include one or more second cameras configured to acquire images of an environment surrounding the head-mounted electronic device. The method may include an operation of identifying a gesture associated with the other wearable device using the second images acquired through the one or more second cameras. The method may include an operation of transmitting a signal to the other wearable device, based on the identification, for activating a mode in which the user interface is controlled according to the user input received for the other wearable device through the communication circuit.

[0248] In one embodiment, the one or more cameras may be one or more first cameras. The images may be first images. The method may further include one or more second cameras configured to acquire images of an environment surrounding the head-wearable electronic device. The method may include an operation of identifying a quick response (QR) code (e.g., QR code (740)) using the second images acquired through the one or more second cameras. The method may include an operation of transmitting, based on the identification, a signal to the other wearable device through the communication circuit to activate a mode in which the user interface is controlled according to the user input received for the other wearable device.

[0249] In one embodiment, the method may include receiving information including the user input from the other wearable device via the communication circuit. The method may include providing a haptic notification in response to the reception.

[0250] In a computer-readable storage medium having one or more programs stored thereon, as described above, the one or more programs may include instructions that, when executed by a head-wearable electronic device having a communication circuit (e.g., the communication circuit (205)), one or more cameras (e.g., one or more first cameras (209)) configured to acquire images of an eye of a user wearing the head-wearable electronic device, and a display assembly (e.g., the display assembly (208)) including at least one display, cause the head-wearable electronic device to display, through the display assembly (e.g., the display assembly (208)), UI object first UI objects (e.g., the first UI objects (130)) as an overlay on an environment (e.g., the environment (150)) surrounding the head-wearable electronic device as seen through the display assembly (e.g., the display assembly (208)). The one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to detect, using images acquired through the one or more cameras (e.g., one or more first cameras (209)), a gaze toward another wearable device (e.g., another wearable device (110)) positioned within the environment (e.g., environment (150)) and worn by a user (e.g., user (120)) of the head-wearable electronic device while displaying the UI object first UI objects (e.g., first UI objects (130)) overlaid on the environment (e.g., environment (150)), which is viewed through the display assembly (e.g., display assembly (208)).The one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to display, on the environment (e.g., the environment (150)), a user interface (e.g., the user interface (490)), including UI objects (e.g., second UI objects (430)), arranged in conjunction with the other wearable device, through the display assembly (e.g., the display assembly (208)), based on the detection. The UI objects (e.g., second UI objects (430)) may include UI objects that each provide functions provided by the first UI objects, respectively. The user interface may be controlled according to a user input received with respect to the other wearable device.

[0251] In one embodiment, the one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to receive, from the other wearable device via the communication circuitry, information about the user input received for the other wearable device while displaying the user interface arranged in conjunction with the other wearable device. The one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to recognize the user input received for the other wearable device as user input received for the user interface.

[0252] In one embodiment, the one or more programs may include instructions that cause the head-wearable electronic device, when executed by the head-wearable electronic device, to detect a gaze toward the user interface using images acquired through the one or more cameras while displaying the user interface arranged in conjunction with the other wearable device. The one or more programs may include instructions that cause the head-wearable electronic device, when executed by the head-wearable electronic device, to receive, from the other wearable device through the communication circuit, information about the user input received for the other wearable device after detecting the gaze while displaying the user interface arranged in conjunction with the other wearable device. The one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to, in response to the receiving, apply the user input received for the other wearable device, and the other user input received for the head-wearable electronic device, as the user input received for the user interface.

[0253] According to one embodiment, the one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to, in response to the user input, move a location of one of the second UI objects from a first location to a second location.

[0254] According to one embodiment, the one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to execute a software application corresponding to one of the second UI objects in response to the user input.

[0255] According to one embodiment, the one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to, in response to the user input, stop displaying at least one of the second UI objects through the display assembly. The one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to, in response to the user input, display a third UI object through the display assembly that is distinct from each of the second UI objects.

[0256] In one embodiment, the one or more cameras may be one or more first cameras. The images may be first images. The head-wearable electronic device may further include one or more second cameras (e.g., one or more second cameras (210)) configured to acquire images of an environment surrounding the head-wearable electronic device. The one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to identify a gesture associated with the other wearable device using second images acquired through the one or more second cameras (e.g., one or more second cameras (210)). The one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to control the user interface based on the identification.

[0257] According to one embodiment, each of the second UI objects may correspond to each of the first UI objects.

[0258] According to one embodiment, the user interface may include a number of second UI objects that are greater than the number of first UI objects displayed through the display assembly, overlapping the environment surrounding the head-wearable electronic device shown through the display assembly.

[0259] In one embodiment, the size of each of the second UI objects may be smaller than the size of each of the first UI objects. The user interface may be in a form in which the second UI objects included in the user interface surround the other wearable device.

[0260] According to one embodiment, the one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to display, through the display assembly, a visual affordance (e.g., visual affordance (730)) corresponding to the other wearable device, as an overlay on the environment surrounding the head-wearable electronic device shown through the display assembly. The one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to detect, using images acquired through the one or more cameras, a gaze directed toward the visual affordance (e.g., visual affordance (730)) displayed through the display assembly. The one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to transmit a signal to the other wearable device for activating a mode in which the user interface is controlled in accordance with the user input received for the other wearable device through the communication circuit based on the detection.

[0261] In one embodiment, the one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to detect, using images acquired through the one or more cameras, a gaze toward another visual affordance (e.g., another visual affordance (830)) displayed through the display assembly while the mode is activated. The one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to, based on the detection, transmit another signal to the other wearable device via the communication circuitry to deactivate the mode in which the user interface is controlled according to the user input received for the other wearable device.

[0262] According to one embodiment, the one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to receive a signal from the other wearable device to activate a mode in which the user interface is controlled in response to the user input received for the other wearable device via the communication circuit.

[0263] In one embodiment, the one or more cameras may be one or more first cameras. The images may be first images. The head-wearable electronic device may further include one or more second cameras configured to acquire images of an environment surrounding the head-wearable electronic device. The one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to identify a gesture associated with the other wearable device using second images acquired through the one or more second cameras. The one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to transmit a signal to the other wearable device for activating a mode in which the user interface is controlled according to the user input received for the other wearable device through the communication circuitry based on the identification.

[0264] In one embodiment, the one or more cameras may be one or more first cameras. The images may be first images. The head-wearable electronic device may further include one or more second cameras configured to acquire images of an environment surrounding the head-wearable electronic device. The one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to identify a quick response (QR) code (e.g., QR code 740) using second images acquired through the one or more second cameras. The one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to transmit a signal to the other wearable device for activating a mode in which the user interface is controlled according to the user input received for the other wearable device through the communication circuitry based on the identification.

[0265] In one embodiment, the one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to receive information including the user input from the other wearable device via the communication circuit. The one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to provide a haptic notification in response to the reception.

[0266] A head-wearable electronic device as described above may include a memory storing instructions. The head-wearable electronic device may include a communication circuit. The head-wearable electronic device may include one or more first cameras configured to acquire images of eyes of a user wearing the head-wearable electronic device. The head-wearable electronic device may include one or more second cameras configured to acquire images of an environment surrounding the head-wearable electronic device. The head-wearable electronic device may include a display assembly including at least one display. The head-wearable electronic device may include at least one processor. The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to display, through the display assembly, first UI objects on each of the first images acquired through the one or more second cameras. The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to, while displaying the first UI objects on each of the first images, detect a gaze directed toward a visual object within each of the first images corresponding to another wearable device positioned in an environment surrounding the head-wearable electronic device and worn by a user of the head-wearable electronic device, using second images acquired through the one or more first cameras.The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to display, through the display assembly, a user interface, based on the detection, on each of the first images, the user interface including second UI objects and arranged in conjunction with the visual objects. The second UI objects may include UI objects that each provide functions provided by the first UI objects. A size of each of the second UI objects may be smaller than a size of each of the first UI objects. The user interface may be controlled according to a user input received for the other wearable device.

[0267] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to receive, via the communication circuitry, from the other wearable device, information about the user input received for the other wearable device while displaying the user interface arranged in conjunction with the other wearable device. The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to recognize the user input received for the other wearable device as user input received for the user interface.

[0268] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to detect, using images acquired through the one or more cameras, a gaze directed toward the user interface while displaying the user interface arranged in conjunction with the other wearable device. The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to receive, through the communication circuitry, information about the user input received for the other wearable device from the other wearable device after detecting the gaze while displaying the user interface arranged in conjunction with the other wearable device. The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to, in response to the receiving, apply the user input received for the other wearable device, and the other user input received for the head-wearable electronic device, as the user input received for the user interface.

[0269] According to one embodiment, the user interface may include a greater number of the second UI objects than the number of the first UI objects displayed through the display assembly.

[0270] In one embodiment, the user interface may be in a form in which the second UI objects included in the user interface surround the other wearable device.

[0271] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to display, through the display assembly, a visual affordance corresponding to the other wearable device. The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to detect, using second images acquired through the one or more first cameras, a gaze directed toward the visual affordance displayed through the display assembly. The instructions, when individually or collectively executed by the at least one processor, may cause the head-wearable electronic device to transmit, based on the detection, a signal to the other wearable device via the communication circuitry to activate a mode in which the user interface is controlled in accordance with the user input received for the other wearable device.

[0272] A method performed by a head-wearable electronic device having a communication circuit as described above, one or more first cameras configured to acquire images of an eye of a user wearing the head-wearable electronic device, one or more second cameras configured to acquire images of an environment surrounding the head-wearable electronic device, and a display assembly including at least one display may include an operation of displaying, through the display assembly, first UI objects on each of the first images acquired through the one or more second cameras. The method may include an operation of detecting, using the second images acquired through the one or more first cameras, while displaying the first UI objects on each of the first images, a gaze directed toward a visual object in each of the first images corresponding to another wearable device positioned in the environment surrounding the head-wearable electronic device and worn by the user of the head-wearable electronic device. The method may include, based on the detection, an operation of displaying, through the display assembly, on each of the first images, a user interface, including second UI objects and arranged in conjunction with the visual object. The second UI objects may include UI objects that each provide functions provided by the first UI objects. The size of each of the second UI objects may be smaller than the size of each of the first UI objects. The user interface may be controlled according to a user input received for the other wearable device.

[0273] In one embodiment, the method may include an operation of receiving, from the other wearable device, information about the user input received for the other wearable device through the communication circuit while displaying the user interface arranged in conjunction with the other wearable device. The method may include an operation of recognizing the user input received for the other wearable device as a user input received for the user interface.

[0274] In one embodiment, the head-wearable electronic device may further include a communication circuit. The method may include an operation of detecting a gaze toward the user interface using images acquired through the one or more cameras while displaying the user interface arranged in conjunction with the other wearable device. The method may include an operation of receiving, from the other wearable device through the communication circuit, information about the user input received for the other wearable device after detecting the gaze while displaying the user interface arranged in conjunction with the other wearable device. The method may include an operation of applying, in response to the reception, the user input received for the other wearable device among the user input received for the other wearable device and the other user input received for the head-wearable electronic device, the user input received for the other wearable device as a user input received for the user interface.

[0275] According to one embodiment, the user interface may include a greater number of the second UI objects than the number of the first UI objects displayed through the display assembly.

[0276] In one embodiment, the user interface may be in a form in which the second UI objects included in the user interface surround the other wearable device.

[0277] In one embodiment, the method may include an operation of displaying a visual affordance corresponding to the other wearable device through the display assembly. The method may include an operation of detecting a gaze directed toward the visual affordance displayed through the display assembly using second images acquired through the one or more first cameras. The method may include an operation of transmitting, based on the detection, a signal to the other wearable device through the communication circuit for activating a mode in which the user interface is controlled according to the user input received for the other wearable device.

[0278] In a computer-readable storage medium having one or more programs stored thereon, as described above, the one or more programs may include instructions that, when executed by a head-wearable electronic device having a communication circuit, one or more first cameras configured to acquire images of an eye of a user wearing the head-wearable electronic device, one or more second cameras configured to acquire images of an environment surrounding the head-wearable electronic device, and a display assembly including at least one display, cause the head-wearable electronic device to display, through the display assembly, first UI objects on each of the first images acquired through the one or more second cameras. The one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to detect, using second images acquired through the one or more first cameras, a gaze directed toward a visual object in each of the first images that corresponds to another wearable device positioned in an environment surrounding the head-wearable electronic device and worn by a user of the head-wearable electronic device, while displaying the first UI objects on each of the first images. The one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to, based on the detection, display, through the display assembly, a user interface, on each of the first images, that includes second UI objects and is arranged in conjunction with the visual objects. The second UI objects may include UI objects that respectively provide functions provided by the first UI objects. The size of each of the second UI objects may be smaller than the size of each of the first UI objects.The above user interface can be controlled based on user input received for the other wearable device.

[0279] In one embodiment, the one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to receive, from the other wearable device via the communication circuitry, information about the user input received for the other wearable device while displaying the user interface arranged in conjunction with the other wearable device. The one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to recognize the user input received for the other wearable device as user input received for the user interface.

[0280] In one embodiment, the one or more programs may include instructions that cause the head-wearable electronic device, when executed by the head-wearable electronic device, to detect a gaze toward the user interface using images acquired through the one or more cameras while displaying the user interface arranged in conjunction with the other wearable device. The one or more programs may include instructions that cause the head-wearable electronic device, when executed by the head-wearable electronic device, to receive, from the other wearable device through the communication circuit, information about the user input received for the other wearable device after detecting the gaze while displaying the user interface arranged in conjunction with the other wearable device. The one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to, in response to the receiving, apply the user input received for the other wearable device, and the other user input received for the head-wearable electronic device, as the user input received for the user interface.

[0281] According to one embodiment, the user interface may include a greater number of the second UI objects than the number of the first UI objects displayed through the display assembly.

[0282] In one embodiment, the user interface may be in a form in which the second UI objects included in the user interface surround the other wearable device.

[0283] In one embodiment, the one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to display, through the display assembly, a visual affordance corresponding to the other wearable device. The one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to detect, using second images acquired through the one or more first cameras, a gaze directed toward the visual affordance displayed through the display assembly. The one or more programs may include instructions that, when executed by the head-wearable electronic device, cause the head-wearable electronic device to transmit, based on the detection, a signal to the other wearable device for activating a mode in which the user interface is controlled according to the user input received for the other wearable device through the communication circuit.

[0284] The devices described above may be implemented as hardware components, software components, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments may be implemented using one or more general-purpose computers or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding to them. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device is sometimes described as being used alone; however, one of ordinary skill in the art will recognize that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, a processing unit may include multiple processors, or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.

[0285] Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing device to perform a desired operation or may independently or collectively command the processing device. The software and / or data may be embodied in any type of machine, component, physical device, computer storage medium, or device for interpretation by the processing device or for providing instructions or data to the processing device. The software may also be distributed over networked computer systems and stored or executed in a distributed manner. The software and data may be stored on one or more computer-readable recording media.

[0286] The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. In this case, the medium may be one that continuously stores a computer-executable program or one that temporarily stores it for execution or download. In addition, the medium may be various recording or storage means in the form of a single or multiple hardware combinations, and is not limited to a medium directly connected to a computer system, but may also be distributed over a network. Examples of the medium may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and those configured to store program commands, including ROM, RAM, and flash memory. In addition, examples of other media may include recording or storage media managed by app stores that distribute applications, sites that supply or distribute various software, servers, etc.

[0287] Although the embodiments described above have been described by way of limited examples and drawings, those skilled in the art will appreciate that various modifications and variations can be made based on the above teachings. For example, appropriate results can still be achieved even if the described techniques are performed in a different order than described, and / or components of the described systems, structures, devices, circuits, etc. are combined or combined in a different manner than described, or are replaced or substituted with other components or equivalents.

[0288] Therefore, other implementations, other embodiments, and equivalents of the claims are also within the scope of the claims described below. According to one embodiment, the method according to the various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0289] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In a head-wearable electronic device, A memory (206) storing instructions and including one or more storage media; Communication circuit (205); One or more cameras (209) configured to acquire images of the eyes of a user (120) wearing the head-wearable electronic device; A display assembly (208) comprising at least one display; and At least one processor (207) comprising processing circuitry, The above instructions, when individually or collectively executed by the at least one processor, Displaying first UI (user interface) objects (130) as an overlay on an environment (150) surrounding the head-wearable electronic device shown through the display assembly (208); While displaying the first UI objects (130) overlaid on the environment (150), detecting a gaze toward another wearable device (110) positioned within the environment (150) shown through the display assembly (208) and worn by a user (120) of the head-wearable electronic device, using images acquired through the one or more cameras (209); and Based on the above detection, causing the head-wearable electronic device to display a user interface (490) including second UI objects (430) and arranged in conjunction with the other wearable device (110) on the environment (150) through the display assembly (208), The above second UI objects (430) are: Includes UI objects that each provide the functions provided by the first UI objects (130), The above user interface (490) is Controlled according to user input received for the above other wearable device (110), Head-mounted electronic devices.

2. In claim 1, The above instructions, when individually or collectively executed by the at least one processor, While displaying the user interface (490) arranged in conjunction with the other wearable device, information about the user input received for the other wearable device (110) is received from the other wearable device (110) through the communication circuit (205), and To recognize the user input received for the other wearable device (110) as a user input received for the user interface (490). Further causing the above head-wearable electronic device, Head-mounted electronic devices.

3. In claim 1, The above instructions, when individually or collectively executed by the at least one processor, While displaying the user interface (490) arranged in conjunction with the other wearable device (110): Using images acquired through one or more of the cameras (209), detecting a gaze toward the user interface (490), and After detecting the above gaze, information about the user input received for the other wearable device (110) is received from the other wearable device (110) through the communication circuit (205), and In response to the above reception, to authorize the user input received for the other wearable device (110) and the other user input received for the head-mounted electronic device as the user input received for the user interface (490), Further causing the above head-wearable electronic device, Head-mounted electronic devices.

4. In claim 1, The above instructions, when individually or collectively executed by the at least one processor, In response to the user input, move the position of one of the second UI objects (430) from the first position to the second position. Further causing the above head-wearable electronic device, Head-mounted electronic devices.

5. In claim 1, The above instructions, when individually or collectively executed by the at least one processor, In response to the user input, execute a software application corresponding to one of the second UI objects (430). Further causing the above head-wearable electronic device, Head-mounted electronic devices.

6. In claim 1, The above instructions, when individually or collectively executed by the at least one processor, In response to the above user input: Stop displaying at least one of the second UI objects (430) through the display assembly (208), and To display a third UI object distinct from each of the second UI objects (430) through the display assembly (208), Further causing the above head-wearable electronic device, Head-mounted electronic devices.

7. In claim 1, further comprising one or more second cameras (210) configured to acquire images of the environment surrounding the head-wearable electronic device; The above instructions, when individually or collectively executed by the at least one processor, Using the second images acquired through the one or more second cameras (210), a gesture related to the other wearable device (110) is identified, and Based on the above identification, to control the user interface (490), Further causing the above head-wearable electronic device, Head-mounted electronic devices.

8. In claim 1, each of the second UI objects (430) Corresponding to each of the above first UI objects (130), Head-mounted electronic devices.

9. In claim 1, the user interface (490) A second UI object (430) that is more than the number of the first UI objects (130) displayed through the display assembly (208) and is superimposed on the environment (150) surrounding the head-wearable electronic device shown through the display assembly (208), Head-mounted electronic devices.

10. In claim 1, the size of each of the second UI objects is: is smaller than the size of each of the above first UI objects, The above user interface (490) is The second UI objects (430) included in the user interface (490) are in a form that surrounds the other wearable device (110). Head-mounted electronic devices.

11. In claim 1, The above instructions, when individually or collectively executed by the at least one processor, Displaying a visual affordance (730) corresponding to the other wearable device (110) overlaid on the environment (150) surrounding the head-wearable electronic device shown through the display assembly (208), Using images acquired through one or more cameras (209), detecting a gaze toward the visual affordance (730) displayed through the display assembly (208), and Based on the above detection, a signal is transmitted to the other wearable device (110) to activate a mode in which the user interface (490) is controlled according to the user input received for the other wearable device (110) through the communication circuit (205). Further causing the above head-wearable electronic device, Head-mounted electronic devices.

12. In claim 11, The above instructions, when individually or collectively executed by the at least one processor, While the above mode is activated, using images acquired through the one or more cameras (209), a gaze toward another visual affordance (830) displayed through the display assembly (208) is detected, and Based on the above detection, another signal is transmitted to the other wearable device (110) to deactivate the mode in which the user interface (490) is controlled according to the user input received for the other wearable device (110) through the communication circuit (205). Further causing the above head-wearable electronic device, Head-mounted electronic devices.

13. In claim 1, The above instructions, when individually or collectively executed by the at least one processor, To receive a signal from the other wearable device (110) to activate a mode in which the user interface (490) is controlled according to the user input received for the other wearable device (110) through the communication circuit (205), Further causing the above head-wearable electronic device, Head-mounted electronic devices.

14. In claim 1, further comprising one or more second cameras (210) configured to acquire images of the environment surrounding the head-wearable electronic device; The above instructions, when individually or collectively executed by the at least one processor, Using the second images acquired through the one or more second cameras (210), a gesture related to the other wearable device (110) is identified, and Based on the above identification, a signal is transmitted to the other wearable device (110) to activate a mode in which the user interface (490) is controlled according to the user input received for the other wearable device (110) through the communication circuit (205). Further causing the above head-wearable electronic device, Head-mounted electronic devices.

15. In a head-wearable electronic device, A memory (206) storing instructions and including one or more storage media; Communication circuit (205); One or more first cameras (209) configured to acquire images of the eyes of a user wearing the head-wearable electronic device; One or more second cameras (210) configured to acquire images of the environment surrounding the head-wearable electronic device; A display assembly (208) comprising at least one display; and At least one processor (207) comprising processing circuitry, The above instructions, when individually or collectively executed by the at least one processor, Displaying first UI objects (130) on each of the first images acquired through the one or more second cameras (210) through the display assembly (208); While displaying the first UI objects (130) on each of the first images, detecting a gaze toward a visual object in each of the first images corresponding to another wearable device (110) positioned in an environment (150) around the head-wearable electronic device and worn by a user (120) of the head-wearable electronic device, using second images acquired through the one or more first cameras (209); and Based on the above detection, a user interface (490) including second UI objects (430) and arranged in conjunction with the visual object is displayed on each of the first images through the display assembly (208). causing the above head-wearable electronic device, The above second UI objects (430) are: Includes UI objects that each provide the functions provided by the first UI objects (130), The above user interface (490) is Controlled according to user input received for the above other wearable device (110), Head-mounted electronic devices.

Images