Wearable device, method, and non-transitory computer-readable storage medium for connecting to external electronic device

Abstract

This wearable device comprises: a memory storing instructions and including one or more storage media; one or more cameras; a display assembly including a display; a communication circuit; and at least one processor including a processing circuit, wherein the instructions, when executed individually or collectively by the at least one processor, may cause the wearable device to: acquire images with the one or more cameras; identify whether a visual object having the shape of an electronic device is included in at least a portion of the images; request, from a server, information about one or more external electronic devices of the type of the electronic device corresponding to the visual object and registered in association with a user account of the wearable device; receive the information from the server; and display a UI object associated with the visual object in a screen on the basis of determining an external electronic device from among the external electronic devices by using the information.

Description

Wearable device, method, and non-transitory computer-readable storage medium for connecting to an external electronic device

[0001] The present disclosure relates to a wearable device, a method, and a non-transitory computer-readable storage medium for connecting to an external electronic device.

[0002] To provide an enhanced user experience, electronic devices are being developed that provide augmented reality (AR) services, which display computer-generated information in conjunction with external objects in the real world. These electronic devices may be wearable devices worn by the user. For example, these electronic devices may be AR glasses and / or head-mounted devices (HMDs).

[0003] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art related to the present disclosure.

[0004] A wearable device is described. The wearable device may include at least one processor comprising a memory storing instructions and including one or more storage media, one or more cameras, a display assembly including a display, communication circuitry, and processing circuitry. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to acquire images of a space in front of the wearable device through the one or more cameras. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify whether a visual object having the shape of an electronic device is included in at least a portion of the images. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to request, through the communication circuitry, from a server, based on identifying that the visual object is included in the at least a portion of the images, information about external electronic devices having a type of electronic device corresponding to the visual object and registered in association with a user account of the wearable device. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to receive, through the communication circuitry, information about one or more external electronic devices from the server. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to determine, from among the one or more external electronic devices, an external electronic device corresponding to the visual object using the information received from the server.The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display, through the display assembly, a user interface (UI) object associated with the visual object based on the determination. The UI object may be available to provide a service related to the external electronic device within the wearable device.

[0005] A method is described. The method may be performed within a wearable device comprising one or more cameras, a display assembly including a display, and communication circuitry. The method may include an operation of acquiring images of a space in front of the wearable device via the one or more cameras. The method may include an operation of identifying whether a visual object having the shape of an electronic device is included in at least a portion of the images. The method may include an operation of requesting, via the communication circuitry, from a server, based on identifying that the visual object is included in the at least a portion of the images, information about external electronic devices having a type of electronic device corresponding to the visual object and registered in association with a user account of the wearable device. The method may include an operation of receiving, via the communication circuitry, information about one or more external electronic devices from the server. The method may include an operation of determining, from among the one or more external electronic devices, an external electronic device corresponding to the visual object using the information received from the server. The method may include an operation of displaying a user interface (UI) object associated with the visual object through the display assembly based on the determination. The UI object may be available to provide a service related to the external electronic device within the wearable device.

[0006] A non-transitory computer-readable storage medium is described. The non-transitory computer-readable storage medium may store one or more programs. The one or more programs may include instructions that, when executed by a wearable device including one or more cameras, a display assembly including a display, and communication circuitry, cause the wearable device to acquire images of a space in front of the wearable device through the one or more cameras. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to identify whether a visual object having the shape of an electronic device is included in at least a portion of the images. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to request, through the communication circuitry, from a server information about external electronic devices having a type of electronic device corresponding to the visual object and registered in association with a user account of the wearable device, based on identifying that the visual object is included in the at least a portion of the images. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to receive, through the communication circuitry, information about one or more external electronic devices from the server. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to determine, from among the one or more external electronic devices, an external electronic device corresponding to the visual object using the information received from the server.The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to display, through the display assembly, a user interface (UI) object associated with the visual object based on the determination. The UI object may be available to provide a service related to the external electronic device within the wearable device.

[0007] Figure 1 illustrates an example environment including a wearable device, multiple external electronic devices, and a server.

[0008] FIG. 2 illustrates an example of a UI object displayed by identifying a visual object having the shape of an electronic device.

[0009] Figure 3a is a simplified block diagram of an exemplary wearable device.

[0010] Figure 3b is a simplified block diagram of an exemplary external electronic device.

[0011] Figure 4 is a simplified block diagram of an exemplary server.

[0012] FIG. 5 is a flowchart illustrating exemplary operations of a wearable device for identifying a visual object having the shape of an electronic device.

[0013] Figure 6 illustrates an example of a visual object having the shape of an electronic device.

[0014] Figure 7 is a signal flow diagram between a wearable device and a server.

[0015] Figure 8 illustrates an example of guidance for turning on an external electronic device.

[0016] Figures 9a and 9b illustrate examples of external electronic devices within a reference distance from a wearable device.

[0017] Figure 10 illustrates an example of the distance between a wearable device and external electronic devices.

[0018] Figures 11a and 11b illustrate examples of external electronic devices determined using sensors of external electronic devices.

[0019] Figure 12a illustrates an example of an external electronic device determined using a visual identifier.

[0020] FIG. 12b illustrates an example of an external electronic device determined using a software application running within the external electronic device.

[0021] Figure 13 illustrates an example of a UI object linked to a visual object.

[0022] Figures 14a and 14b illustrate examples of other UI objects for external electronic devices that are not registered to a user account.

[0023] Figure 15 is a signal flow diagram between a wearable device and a server.

[0024] FIG. 16 is a block diagram of an electronic device within a network environment according to various embodiments.

[0025] Figure 17a shows an example of a perspective view of a wearable device.

[0026] FIG. 17b illustrates an example of one or more hardware devices arranged within a wearable device.

[0027] Figures 18a and 18b show an example of the appearance of a wearable device.

[0028] Figure 19 shows an example of a block diagram of a wearable device.

[0029] Fig. 20 shows an example of a block diagram of an electronic device for displaying an image in virtual space.

[0030] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0031] Figure 1 illustrates an example environment including a wearable device, multiple external electronic devices, and a server.

[0032] Referring to FIG. 1, a wearable device (100) may be described as a head-mounted display (HMD) device, a headgear electronic device, a glasses-type (or goggle-type) electronic device, a video see-through (VST) device, an extended reality (XR) device, a virtual reality (VR) device, and / or an augmented reality (AR) device that can be worn on a user's head. An example of the structure of a wearable device (100) that can be worn on a user's head is described with reference to FIGS. 17A, 17B, 18A, and / or 18B. For example, the wearable device (100) may be used to provide augmented reality (AR) and / or mixed reality (MR).

[0033] For example, the wearable device (100) may include a communication circuit (e.g., the communication circuit (320) of 3a). For example, the wearable device (100) may be connected to a server (120) using the communication circuit. For example, the server (120) may provide an IoT (Internet of Things) service. For example, the server (120) may store user account information of the wearable device (100) and information about a plurality of external electronic devices (110) in a memory (e.g., memory (410)). For example, the wearable device (100) may request information about a plurality of external electronic devices (110) registered in connection with a user account of the wearable device (100) from the server (120) through the communication circuit. For example, the user account may include a user account for an IoT service.

[0034] For example, the server (120) may include a communication circuit (e.g., the communication circuit (420) of FIG. 4). For example, the server (120) may receive a request for information about a plurality of external electronic devices (110) registered in conjunction with a user account of the wearable device (100) from the wearable device (100) through the communication circuit. For example, the server (120) may transmit information about the plurality of external electronic devices (110) to the wearable device (100) through the communication circuit. For example, the wearable device (100) may receive information about the plurality of external electronic devices (110) from the server (120) through the communication circuit.

[0035] For example, the server (120) may be connected to a plurality of external electronic devices (110) through the communication circuit. For example, the wearable device (100) may be directly connected to a plurality of external electronic devices (110) or may be connected through the server (120). For example, the plurality of external electronic devices (110) may be directly connected to the wearable device (100) or may be connected through the server (120), thereby providing services related to the plurality of external electronic devices (110) to the wearable device (100). For example, the wearable device (100) may transmit a signal to the plurality of external electronic devices (110) directly through the communication circuit (or through the server (120)) to cause the plurality of external electronic devices (110) to perform a function within the plurality of external electronic devices (110).

[0036] For example, the plurality of external electronic devices (110) may include a communication circuit (e.g., the communication circuit (325) of FIG. 3B). For example, the plurality of external electronic devices (110) may directly (or through the server (120)) receive a signal through the communication circuit that causes the plurality of external electronic devices (110) to perform a function within the plurality of external electronic devices (110). For example, the plurality of external electronic devices (110) may transmit data acquired based on performing the function to the wearable device (100) through the communication circuit. For example, the wearable device (100) may receive data acquired based on performing the function from the plurality of external electronic devices (110) using the communication circuit.

[0037] For example, a wearable device (100) may display a UI object to be directly connected to a plurality of external electronic devices (110) or to be connected via a server (120). Displaying a UI object is exemplified in the description of FIG. 2.

[0038] FIG. 2 illustrates an example of a UI object displayed by identifying a visual object having the shape of an electronic device.

[0039] Referring to FIG. 2, the wearable device (200) may include one or more cameras (e.g., one or more cameras (330) of FIG. 3A). For example, a state (210) may be described as a state in which a UI object (220) is displayed. For example, within the state (210), the wearable device (200) may acquire images of a space in front of the wearable device (200) using the one or more cameras. For example, an external electronic device may be positioned within the space in front of the wearable device (200).

[0040] For example, the wearable device (200) may include a display assembly (e.g., the display assembly (340) of FIG. 3A). For example, the wearable device (200) may display a screen on the display assembly using at least some of the images. For example, the wearable device (200) may identify whether a visual object (215) having the shape of an electronic device is included in at least some of the images while the screen is displayed. For example, the visual object (215) may correspond to an external electronic device. For example, the wearable device (200) may display the visual object (215) on the screen using at least some of the images including the visual object (215).

[0041] For example, the wearable device (200) may display a UI object (220) associated with the visual object (215) on the screen based on identifying that the visual object (215) is included within at least a portion of the images. For example, the UI object (220) may be available to provide a service related to an external electronic device within the wearable device (200). For example, the wearable device (200) may be connected to the external electronic device based on an input to the UI object (220).

[0042] For example, since the external electronic device is not registered to the user account of the wearable device (200), the user (205) may not have an intention to connect the wearable device (200) with the external electronic device. For example, the wearable device (200) may display a UI object (220) on the screen regardless of whether the external electronic device is registered to the user account of the wearable device (200), and thus, the user (205) who does not have an intention to connect the wearable device (200) with the external electronic device may feel inconvenienced. For example, a method may be required to resolve the inconvenience of the user (205) caused by the UI object (220) displayed on the screen regardless of whether the external electronic device is registered to the user account of the wearable device (200).

[0043] For example, to resolve such inconvenience, the wearable device (200) may display a UI object (220) on the screen based on an external electronic device registered in association with a user account of the wearable device (200). For example, in order to display the UI object (220) on the screen based on an external electronic device registered in association with a user account of the wearable device (200), information about the external electronic devices registered in association with the user account may be used. For example, the wearable device (200) may determine whether an external electronic device is registered to a user account by using information about the external electronic devices registered in association with the user account.

[0044] The wearable device (200) may perform the operations exemplified in the descriptions of FIGS. 5 through 15 to determine whether an external electronic device is registered to a user account. The wearable device (200) may include components for performing the operations. The components may be exemplified in the description of FIG. 3A.

[0045] Figure 3a is a simplified block diagram of an exemplary wearable device.

[0046] Referring to FIG. 3A, the wearable device (100) may be described as a head-mounted display (HMD) device, a headgear electronic device, a glasses-type (or goggle-type) electronic device, a video see-through (VST) device, an extended reality (XR) device, a virtual reality (VR) device, and / or an augmented reality (AR) device that can be worn on a user's head. An example of the structure of the wearable device (100) that can be worn on a user's head is described with reference to FIGS. 17A, 17B, 18A, and / or 18B. The wearable device (100) may include at least a part of the electronic device (1600) of FIG. 16, or may correspond to at least a part of the electronic device (1600) of FIG. 16. A wearable device (100) may include at least one processor (300), memory (310), communication circuitry (320), one or more cameras (330), and a display assembly (340).

[0047] At least one processor (300) may include processing circuitry. For example, at least one processor (300) may include a central processing unit (CPU) (e.g., including processing circuitry). For example, at least one processor (300) may include a graphic processing unit (GPU) (e.g., including processing circuitry) and a neural processing unit (NPU) (e.g., including processing circuitry). For example, at least one processor (300) may be configured to control a memory (310), a communication circuit (320), one or more cameras (330), and a display assembly (340). At least one processor (300) may be configured to individually or collectively execute instructions stored in the memory (310) to cause the wearable device (100) (or the wearable device (200)) to perform at least some of the operations illustrated in the descriptions of FIGS. 1 and 2 . At least one processor (300) may be configured to individually or collectively execute instructions stored in the memory (310) to cause the wearable device (100) to perform at least some of the operations exemplified in the descriptions of FIGS. 5 through 15.

[0048] The memory (310) may include one or more storage media. For example, the memory (310) may store various data used by at least one component of the wearable device (100) (e.g., at least one processor (300), a communication circuit (320), one or more cameras (330), and / or a display assembly (340)). For example, the data may include input data or output data for software and commands related thereto. The memory (310) may include volatile memory or non-volatile memory.

[0049] The communication circuit (320) may include hardware components for supporting transmission and / or reception of signals between the wearable device (100) and a plurality of external electronic devices (110). The communication circuit (320) may include hardware components for supporting transmission and / or reception of signals between the wearable device (100) and the server (120). The communication circuit (320) may include, for example, at least one of a modem, an antenna, and an optical / electronic (O / E) converter. The communication circuit (320) may support transmission and / or reception of electrical signals based on various types of protocols, such as Ethernet, a local area network (LAN), a wide area network (WAN), wireless fidelity (WiFi), Bluetooth, Bluetooth low energy (BLE), zigbee, long term evolution (LTE), and 5G new radio (NR). For example, the communication circuit (320) may be used to transmit a request to the server (120). For example, the communication circuit (320) may be used to receive information from the server (120) (or an external electronic device). For example, the communication circuit (320) may be used to transmit signals directly to external electronic devices (or through the server (120)). For example, the communication circuit (320) may be used to receive data directly from external electronic devices (or through the server (120)).

[0050] The one or more cameras (330) may include one or more optical sensors (e.g., a charged coupled device (CCD) sensor, a complementary metal oxide semiconductor (CMOS) sensor) that generate an electrical signal representing the color and / or brightness of light. For example, the one or more cameras (330) may be described as an image sensor. For example, the one or more cameras (330) may be available to acquire images of the space in front of the wearable device (100) (or the surrounding environment). For example, at least some of the one or more cameras (330) may have a field of view (FOV) corresponding to the FOV of the user's eye. For example, the FOV of some of the one or more cameras (330) may be different from the FOV of other some of the one or more cameras (330).

[0051] The display assembly (340) may be configured to display a screen, a visual object, and / or a UI object. For example, the display assembly (340) may include at least one display. For example, the display assembly (340) may be used to display a visual object corresponding to an external electronic device on the screen. For example, the display assembly (340) may be used to display a UI object associated with a visual object on the screen.

[0052] The plurality of external electronic devices (110) (or external electronic devices) exemplified in the descriptions of FIGS. 1 and 2 may perform at least some of the operations exemplified in the descriptions of FIGS. 5 to 15 . For example, the operations exemplified in the descriptions of FIGS. 5 to 15 may be caused by (or within) the external electronic device under the control of at least one processor. For example, the external electronic device may include components for performing these operations. The components of the external electronic device are exemplified in the description of FIG. 3B .

[0053] Figure 3b is a simplified block diagram of an exemplary external electronic device.

[0054] Referring to FIG. 3B, the external electronic device (301) may be one of various forms of mobile devices, such as smartphones having various form factors (e.g., bar-type smartphones, foldable-type smartphones, or rollable-type smartphones), tablets, wearable devices, cellular phones, laptops, and / or other similar computing devices. For example, the external electronic device (301) may be included in a plurality of external electronic devices (110). The external electronic device (301) may include at least a portion of the electronic device (1604) of FIG. 16 or may correspond to at least a portion of the electronic device (1604) of FIG. 16. The external electronic device (301) may include at least one processor (305), a memory (315), a communication circuit (325), a sensor (335), and a display (345).

[0055] At least one processor (305) may include processing circuitry. For example, at least one processor (305) may include a central processing unit (CPU) (e.g., including processing circuitry). For example, at least one processor (305) may include a graphic processing unit (GPU) (e.g., including processing circuitry) and a neural processing unit (NPU) (e.g., including processing circuitry). For example, at least one processor (305) may be configured to control memory (315), communication circuitry (325), sensor (335), and display (345). At least one processor (305) may be configured to individually or collectively execute instructions stored in memory (315) to cause an external electronic device to perform at least some of the operations illustrated in the descriptions of FIGS. 1 and 2 . At least one processor (305) may be configured to individually or collectively execute instructions stored in the memory (315) to cause the external electronic device (301) to perform at least some of the operations exemplified in the descriptions of FIGS. 5 through 15.

[0056] The memory (315) may include one or more storage media. For example, the memory (315) may store various data used by at least one component of the external electronic device (301) (e.g., at least one processor (305), a communication circuit (325), a sensor (335), and / or a display (345)). For example, the data may include input data or output data for software and commands related thereto. The memory (315) may include volatile memory or non-volatile memory.

[0057] The communication circuit (325) may include hardware components for supporting transmission and / or reception of signals between the external electronic device (301) and the wearable device (100). The communication circuit (325) may include hardware components for supporting transmission and / or reception of signals between the external electronic device (301) and the server (120). The communication circuit (325) may include, for example, at least one of a modem, an antenna, and an optical / electronic (O / E) converter. The communication circuit (325) may support transmission and / or reception of electrical signals based on various types of protocols, such as Ethernet, a local area network (LAN), a wide area network (WAN), wireless fidelity (WiFi), Bluetooth, Bluetooth low energy (BLE), zigbee, long term evolution (LTE), and 5G new radio (NR). For example, the communication circuit (325) may be used to receive an audio signal from the server (120) (or an external electronic device). For example, the communication circuit (325) may be used to connect directly with the wearable device (100) (or via the server (120)). For example, the communication circuit (325) may be used to receive a signal from the wearable device (100) (or the server (120)) that causes the external electronic device (301) to activate the sensor (335). For example, the communication circuit (325) may be used to transmit sensing data to the wearable device (100).

[0058] The sensor (335) may be configured to acquire sensing data based on a user gesture. For example, the sensor (335) may be activated based on a signal received from the wearable device (100) that causes the external electronic device (301) to activate the sensor (335). As a non-limiting example, the sensor (335) may include an acceleration sensor, a gyro sensor, a touch sensor, a barometric pressure sensor, an infrared (IR) sensor, and / or an image sensor. However, the present invention is not limited thereto.

[0059] The display (345) may be configured to display a visual identifier. For example, the display (345) may be configured to display a screen provided from a software application running within the external electronic device (301).

[0060] The server (120) illustrated in the description of FIG. 1 may execute at least some of the operations illustrated in the descriptions of FIGS. 5 to 15 . For example, the operations illustrated in the descriptions of FIGS. 5 to 15 may be caused by (or within) the server (120) under the control of at least one processor. For example, the server (120) may include components for executing these operations. The components of the server are illustrated in the description of FIG. 4 .

[0061] Figure 4 is a simplified block diagram of an exemplary server.

[0062] Referring to FIG. 4, the server (120) may be described as one or more personal computers (PCs) and / or workstations. The server (120) may include at least a portion of the server (1608) of FIG. 16, or may correspond to at least a portion of the server (1608) of FIG. 16. For example, the server (120) may provide IoT services. For example, the server (120) may store information about a user account of a wearable device (100) and a plurality of external electronic devices (110) registered in connection with the user account, and may calculate the generated data. The server (120) may include at least one processor (400), a memory (410), and a communication circuit (420).

[0063] At least one processor (400) may include processing circuitry. For example, at least one processor (400) may include a central processing unit (CPU) (e.g., including processing circuitry). For example, at least one processor (400) may include a graphic processing unit (GPU) (e.g., including processing circuitry) and a neural processing unit (NPU) (e.g., including processing circuitry). For example, at least one processor (400) may be configured to control memory (410) and communication circuitry (420). At least one processor (400) may be configured to individually or collectively execute instructions stored in memory (410) to cause the server (120) to perform at least some of the operations illustrated in the description of FIG. 1. At least one processor (400) may be configured to individually or collectively execute instructions stored in memory (410) to cause the server (120) to perform at least some of the operations illustrated in the descriptions of FIGS. 5 through 15.

[0064] The memory (410) may include one or more storage media. For example, the memory (410) may store various data used by at least one component of the server (120) (e.g., the memory (410) and the communication circuit (420)). For example, the data may include input data or output data for software and commands related thereto. The memory (410) may include volatile memory or non-volatile memory.

[0065] The communication circuit (420) may include hardware components for supporting transmission and / or reception of signals between the server (120) and the wearable device (100). The communication circuit (420) may include hardware components for supporting transmission and / or reception of signals between the server (120) and the external electronic device (301). The communication circuit (420) may include, for example, at least one of a modem, an antenna, and an optical / electronic (O / E) converter. The communication circuit (420) may support transmission and / or reception of electrical signals based on various types of protocols, such as Ethernet, a local area network (LAN), a wide area network (WAN), wireless fidelity (WiFi), Bluetooth, Bluetooth low energy (BLE), zigbee, long term evolution (LTE), and 5G new radio (NR).

[0066] For example, the communication circuit (420) may be used to receive a request from the wearable device (100). For example, the communication circuit (420) may be used to transmit information about a plurality of external electronic devices (110) to the wearable device (100). For example, the communication circuit (420) may be used to connect the wearable device (100) and the external electronic device (301).

[0067] FIG. 5 is a flowchart illustrating exemplary operations of a wearable device for identifying a visual object having the shape of an electronic device.

[0068] Referring to FIG. 5, in operation 500, at least one processor (300) may acquire images of an environment around a wearable device (100) using one or more cameras (330). For example, at least some of the one or more cameras (330) have an FOV corresponding to an FOV of a user's eyes, so that at least one processor (300) may acquire images of a space in front of the wearable device (100) through the one or more cameras (330).

[0069] In operation 510, at least one processor (300) may display a screen on a display assembly (340) using at least a portion of the images. For example, at least one processor (300) may provide a 3D (three-dimensional) space through the display assembly (340). For example, at least one processor (300) may display a screen using at least a portion of the images within the 3D space.

[0070] In another embodiment, a wearable device (100) described as a glasses-type (or goggle-type) electronic device and / or an augmented reality (AR) device may refrain from (or stop, or not display) displaying a screen on a display assembly (340) using at least some of the images. For example, at least one processor (300) may perform operation 520 while not displaying a screen on the display assembly (340) using at least some of the images.

[0071] In operation 520, at least one processor (300) may identify whether a visual object having the shape of an electronic device is included in at least some of the images. For example, when an external electronic device (301) is positioned within a space in front of the wearable device (100), a visual object corresponding to the external electronic device (301) may be included in at least some of the images. The inclusion of a visual object having the shape of an electronic device in at least some of the images is exemplified in the description of FIG. 6 .

[0072] Figure 6 illustrates an example of a visual object having the shape of an electronic device.

[0073] Referring to FIG. 6, a state (600) may be described as a state in which a visual object (215) is displayed within a screen displayed on a display assembly (340). For example, within the state (600), an external electronic device (301) may be positioned within a space in front of a wearable device (100). For example, at least one processor (300) may display a screen using at least a portion of images including a visual object (215) corresponding to the external electronic device (301) positioned within the space in front of the wearable device (100).

[0074] For example, at least one processor (300) can identify whether the visual object (215) has the shape of an electronic device. For example, by identifying whether the visual object (215) has the shape of an electronic device, at least one processor (300) can identify whether an external electronic device (301) located within a space in front of the wearable device (100) is an electronic device. For example, by identifying that the external electronic device (301) is an electronic device, at least one processor (300) can determine whether to display a UI object available for providing a service related to the external electronic device (301) in conjunction with the visual object (215).

[0075] For example, at least one processor (300) may identify a type of an external electronic device (301) using at least some of the images. For example, the acquired type of the external electronic device (301) may be used to determine whether to display the UI object associated with the visual object (215). For example, at least one processor (300) may perform operations (e.g., operations 710 to 750) of FIG. 7 based on identifying that a visual object (215) having the shape of an electronic device is included in at least some of the images. Operations performed based on identifying that a visual object (215) having the shape of an electronic device is included in at least some of the images are exemplified in the description of FIG. 7.

[0076] Figure 7 is a signal flow diagram between a wearable device and a server.

[0077] Referring to FIG. 7, at operation 700, at least one processor (300) may identify whether a visual object having the shape of an electronic device (e.g., the visual object (215) of FIG. 6) is included within at least a portion of the images. For example, operation 700 may correspond to operation 520 of FIG. 5.

[0078] In another embodiment, the wearable device (100) may further include one or more other cameras arranged to face the eyes of the user (605) of the wearable device (100) when worn. For example, at least one processor (300) may identify the gaze of the user (605) using the one or more other cameras while the screen is displayed. For example, at least one processor (300) may identify that the gaze of the user (605) is directed toward the visual object (215). For example, by the gaze of the user (605) being directed toward the visual object (215), the user (605) may have an intention to connect (or link) the wearable device (100) with an external electronic device (301) corresponding to the visual object (215). For example, at least one processor (300) may perform operations 710 to 750 based on identifying a gaze of a user (605) toward a visual object (215).

[0079] At operation 710, at least one processor (300) may request information about external electronic devices from a server (120) via a communication circuit (320) based on identifying that a visual object having the shape of an electronic device is included in at least some of the images. At least one processor (400) may receive the request from the wearable device (100) via a communication circuit (420).

[0080] For example, the external electronic devices may be included in the plurality of external electronic devices (110) of FIG. 1. For example, the external electronic devices may have a type of electronic device corresponding to the visual object. As a non-limiting example, the external electronic devices may have the type of electronic device as described above, since the visual object corresponds to an electronic device such as a smartphone, tablet, wearable device, smartwatch, cellular phone, laptop, desktop, or other similar computing devices. However, the present invention is not limited thereto.

[0081] For example, the external electronic devices may be included in electronic devices registered in connection with a user account of the wearable device (100). For example, at least one processor (300) may identify a user account of the wearable device (100) based on unique information of a user wearing the wearable device (100) (e.g., the user's iris information or the user's password). For example, the user account may be used within the wearable device (100). For example, the user account may be related to an IoT service provided through the server (120).

[0082] In operation 720, at least one processor (400) may identify a user account of the wearable device (100) based on receiving the request. For example, at least one processor (400) may identify external electronic devices registered to the user account of the wearable device (100).

[0083] For example, external electronic devices registered to a user account of a wearable device (100) may be registered in association with the user account of the wearable device (100) by previously being connected to the wearable device (100) or by being linked with the wearable device (100) and providing services related to the external electronic devices within the wearable device (100). For example, external electronic devices registered to a user account of a wearable device (100) may be registered in association with the user account by logging in to the user account within the external electronic devices.

[0084] For example, at least one processor (400) may identify the type of external electronic device (301) corresponding to the visual object based on receiving the request. For example, at least one processor (400) may identify one or more external electronic devices having the type of external electronic device (301) among external electronic devices registered to a user account of the wearable device (100).

[0085] In operation 730, at least one processor (400) may transmit information about one or more identified external electronic devices to the wearable device (100) via the communication circuit (420). For example, at least one processor (300) may receive information about one or more external electronic devices from the server (120) via the communication circuit (320).

[0086] In operation 740, at least one processor (300) may, based on receiving information about one or more external electronic devices from the server (120), use the information about the one or more external electronic devices to identify whether the external electronic device (301) is registered as linked to a user account of the wearable device (100). For example, the at least one processor (300) may use the information about the one or more external electronic devices to determine the external electronic device (301) from among the one or more external electronic devices to identify whether the external electronic device (301) is registered as linked to a user account of the wearable device (100).

[0087] For example, at least one processor (300) may determine an external electronic device (301) from among one or more external electronic devices by controlling one or more external electronic devices using information about the one or more external electronic devices. For example, if the external electronic device (301) from among one or more external electronic devices is turned off, the at least one processor (300) may not be able to control the external electronic device (301), thereby causing an error in determining the external electronic device (301) from among the one or more external electronic devices. For example, the at least one processor (300) may request a user to turn on the external electronic device (301). Requesting the user to turn on the external electronic device (301) is exemplified in the description of FIG. 8.

[0088] Figure 8 illustrates an example of guidance for turning on an external electronic device.

[0089] Referring to FIG. 8, a state (800) may be described as a state in which an external electronic device (301) is turned off. For example, within the state (800), at least one processor (300) may display a screen including a visual object (215) on a display assembly (340) using images of a space in front of the wearable device (100) acquired through one or more cameras (330). For example, the visual object (215) may correspond to the external electronic device (301). For example, when the external electronic device (301) is turned off, the visual object (215) may have the shape of the turned-off electronic device.

[0090] For example, at least one processor (300) may identify that a visual object (215) having the shape of a turned-off electronic device is included in at least some of the images. For example, at least one processor (300) may display guidance (805) on the screen to turn on the external electronic device (301) based on identifying a visual object (215) having the shape of a turned-off electronic device included in at least some of the images. For example, the guidance (805) may be displayed in association with the visual object (215).

[0091] For example, the user (705) may turn on the external electronic device (301) according to the guidance (805). For example, at least one processor (300) may use the external electronic device (301) turned on by the user to identify whether the external electronic device (301) is registered as linked to the user account of the wearable device (100). For example, at least one processor (300) may use the turned-on external electronic device (301) to identify whether external electronic devices are within a reference distance from the wearable device (100). Whether the external electronic devices are within a reference distance is exemplified in the description of FIGS. 9A and 9B.

[0092] Figures 9a and 9b illustrate examples of external electronic devices within a reference distance from a wearable device.

[0093] Referring to FIGS. 9A and 9B , at least one processor (300) may use information about one or more external electronic devices to identify an external electronic device included in the same network as the wearable device (100) among the one or more external electronic devices. For example, at least one processor (300) may use information about the one or more external electronic devices to measure a distance between the wearable device (100) and the one or more external electronic devices.

[0094] For example, at least one processor (300) can identify (or measure) a distance between the wearable device (100) and one or more external electronic devices by using the strength of a wireless signal between the wearable device (100) and one or more external electronic devices. For example, the strength of a wireless signal between the wearable device (100) and one or more external electronic devices can include a received signal strength indicator (RSSI). For example, the distance between the wearable device (100) and one or more external electronic devices can be obtained by applying an RSSI value between the wearable device (100) and one or more external electronic devices to the following mathematical expression 1.

[0095]

[0096] In mathematical expression 1, n represents a constant reflecting the characteristics of the signal intensity that is attenuated depending on the environment, d represents a distance between the wearable device (100) and one or more external electronic devices, and A may represent an RSSI value measured at a distance of 1 m from the wearable device (100). For example, a value obtained by applying an RSSI value between the wearable device (100) and one or more external electronic devices to mathematical expression 1 (e.g., the d value in mathematical expression 1) may represent a distance between the wearable device (100) and one or more external electronic devices.

[0097] For example, at least one processor (300) can identify a distance between the wearable device (100) and one or more external electronic devices and a direction of the external electronic devices with respect to the wearable device (100) through transmission and reception of high-frequency signals according to a WiFi communication standard (e.g., 802.11.ad communication standard) when the wearable device (100) and one or more external electronic devices are located within the same WiFi environment.

[0098] For example, at least one processor (300) can identify (or measure) an angle of a wireless signal that the wearable device (100) receives from one or more external electronic devices using an AOA (angle of arrival) of BLE (Bluetooth low energy). For example, at least one processor (300) can identify (or measure) an angle of a wireless signal that the wearable device (100) transmits to one or more external electronic devices using an AOD (angle of departure) of BLE. For example, at least one processor (300) can identify locations of one or more external electronic devices using an AOA and an AOD of BLE.

[0099] Referring to FIG. 9A, a state (900) can be described as a state in which an external electronic device (301) among one or more external electronic devices (301, 110-1, 110-2) is located in an area (905) within a reference distance from a wearable device (100). For example, in the state (900), at least one processor (300) can identify whether one or more external electronic devices (301, 110-1, 110-2) is located in an area (905) within a reference distance from a wearable device (100) by measuring a distance between the wearable device (100) and one or more external electronic devices (301, 110-1, 110-2) and / or positions of one or more external electronic devices (301, 110-1, 110-2).

[0100] For example, only one external electronic device (301) among one or more external electronic devices (301, 110-1, 110-2) may be located in an area (905) within a reference distance from the wearable device (100). For example, at least one processor (300) may determine the external electronic device (301) among one or more external electronic devices (301, 110-1, 110-2) based on identifying that only one external electronic device (301) among one or more external electronic devices (301, 110-1, 110-2) is located in an area (905) within a reference distance from the wearable device (100). For example, at least one processor (300) can identify that an external electronic device (301) is registered as linked to a user account by determining the external electronic device (301) from among one or more external electronic devices (301, 110-1, 110-2).

[0101] Referring to FIG. 9B, a state (920) can be described as a state in which a plurality of external electronic devices (301, 110-1, 110-2) among one or more external electronic devices (301, 110-2) are located in an area (905) within a reference distance from the wearable device (100). For example, in the state (920), at least one processor (300) can identify whether one or more external electronic devices (301, 110-1, 110-2) are located in an area (905) within a reference distance from the wearable device (100) by measuring a distance between the wearable device (100) and one or more external electronic devices (301, 110-1, 110-2) and / or a position of one or more external electronic devices (301, 110-1, 110-2).

[0102] For example, among one or more external electronic devices (301, 110-1, 110-2), a plurality of external electronic devices (301, 101-2) may be located in an area (905) within a reference distance from the wearable device (100). For example, at least one processor (300) may determine the plurality of external electronic devices (301, 101-2) among one or more external electronic devices (301, 110-1, 110-2) by determining that the plurality of external electronic devices (301, 101-2) are located in an area (905) within a reference distance from the wearable device (100). For example, at least one processor (300) may identify that the plurality of external electronic devices (301, 101-2) are registered as linked to a user account. For example, at least one processor (300) may be unable to determine which external electronic device the user (605) intends to connect with the wearable device (100) by identifying that a plurality of external electronic devices (301, 101-2) are registered as linked to the user account. For example, at least one processor (300) may be required to determine one external electronic device (301) from among a plurality of external electronic devices (101-2, 301) located in an area (905) within a reference distance from the wearable device (100). Determining one external electronic device (301) from among a plurality of external electronic devices (101-2, 301) is exemplified in the description of FIG. 10.

[0103] Figure 10 illustrates an example of the distance between a wearable device and external electronic devices.

[0104] Referring to FIG. 10, a state (1000) can be described as a state in which one or more external electronic devices (301, 110-1, 110-2) are located in an area within a reference distance from a wearable device (100). For example, within the state (1000), at least one processor (300) can identify (or measure) distances (1005-1, 1005-2, 1005-3) between the wearable device (100) and one or more external electronic devices (301, 110-1, 110-2) by using information about the one or more external electronic devices (301, 110-1, 110-2) according to the method exemplified in the description of FIGS. 9A and 9B.

[0105] For example, among one or more external electronic devices (301, 110-1, 110-2), the external electronic device (301) may be positioned relatively close to the wearable device (100). For example, at least one processor (300) may identify a distance (1005-1) between the wearable device (100) and the external electronic device (301) that is relatively short among the distances (1005-1, 1005-2, 1005-3) between the wearable device (100) and the one or more external electronic devices (301, 110-1, 110-2), thereby identifying that the external electronic device (301) is positioned relatively close to the wearable device (100).

[0106] For example, at least one processor (300) can determine an external electronic device (301) located relatively close to the wearable device (100) from among one or more external electronic devices (301, 110-1, 110-2). For example, at least one processor (300) can identify that the external electronic device (301) is registered as linked to a user account by determining the external electronic device (301) from among one or more external electronic devices (301, 110-1, 110-2).

[0107] In another embodiment, at least one processor (300) can identify (or measure) the strength of a wireless signal between the wearable device (100) and one or more external electronic devices (301, 110-1, 110-2). For example, the strength of the wireless signal between the wearable device (100) and one or more external electronic devices (301, 110-1, 110-2) can include RSSI. For example, the strength of the wireless signal between the wearable device (100) and one or more external electronic devices (301, 110-1, 110-2) can be related to a distance between the wearable device (100) and one or more external electronic devices (301, 110-1, 110-2).

[0108] For example, at least one processor (300) can identify the strength of a wireless signal between the wearable device (100) and one or more external electronic devices (301, 110-1, 110-2), thereby identifying that the strength of the wireless signal between the wearable device (100) and the external electronic device (301) is stronger than the strength of the wireless signal between the wearable device (100) and the external electronic devices (110-1, 110-2).

[0109] For example, at least one processor (300) can determine an external electronic device (301) having a relatively strong wireless signal from a wearable device (100) among one or more external electronic devices (301, 110-1, 110-2). For example, at least one processor (300) can identify that the external electronic device (301) is registered as linked to a user account by determining the external electronic device (301) among one or more external electronic devices (301, 110-1, 110-2).

[0110] For example, at least one processor (300) can use a sensor (335) of an external electronic device (301) to determine the external electronic device (301) from among one or more external electronic devices (301, 110-1, 110-2). The use of the sensor (335) of the external electronic device (301) to determine the external electronic device (301) from among one or more external electronic devices (301, 110-1, 110-2) is exemplified within the description of FIGS. 11A and 11B.

[0111] Figures 11a and 11b illustrate examples of external electronic devices determined using sensors of external electronic devices.

[0112] Referring to FIGS. 11A and 11B , at least one processor (300) may use information about one or more external electronic devices received from the server (120) to transmit a signal to one or more external electronic devices via the communication circuit (320) to cause the one or more external electronic devices to activate a sensor of the one or more external electronic devices. For example, the one or more external electronic devices may receive a signal from the wearable device (100) via the communication circuit (e.g., the communication circuit (325) of the external electronic device (301)) to cause the one or more external electronic devices to activate a sensor of the one or more external electronic devices. For example, the one or more external electronic devices may activate a sensor of the one or more external electronic devices based on receiving the signal.

[0113] Referring to FIG. 11A, a state (1100) may be described as a state in which a sensor (335) of an external electronic device (301), including a laptop, desktop, and / or tablet, is activated. For example, the sensor (335) may include an image sensor or a camera. For example, within the state (1100), at least one processor (300) may display guidance (1105) on a screen to perform a user gesture associated with the external electronic device (301) based on transmitting a signal to one or more external electronic devices, the signal causing the one or more external electronic devices to activate a sensor of the one or more external electronic devices. For example, the guidance (1105) may be displayed in conjunction with a visual object (215).

[0114] For example, the user (605) may perform a user gesture associated with the external electronic device (301) according to the guidance (1105). As a non-limiting example, the guidance (1105) may include text instructing the user to bring the user's hand (1110) closer to the external electronic device (301), such that the user (605) may move the hand (1110) toward the external electronic device (301). The user gesture may include a gesture in which the user (605) moves the hand (1110) toward the external electronic device (301). When a user (605) moves his / her hand (1110) in the direction of an external electronic device (301), at least one processor (300) can display an object (1115) corresponding to the hand (1110) of the user (605) moving in the direction of the visual object (215) on the screen. When the user (605) moves his / her hand (1110) in the direction of the external electronic device (301), at least one processor (305) can identify the user gesture through the sensor (335). At least one processor (305) can obtain images of the hand (1110) of the user (605) through the sensor (335). At least one processor (305) can identify the user gesture using the images of the hand (1110) of the user (605). However, the present invention is not limited thereto.

[0115] For example, at least one processor (305) can obtain sensing data about a user gesture by identifying the user gesture through the sensor (335). For example, at least one processor (305) can transmit the sensing data about the user gesture to the wearable device (100) through the communication circuit (325) based on obtaining the sensing data about the user gesture. For example, at least one processor (300) can receive sensing data about the user gesture from an external electronic device (301) through the communication circuit (320). For example, at least one processor (300) can identify the user gesture using the sensing data about the user gesture.

[0116] For example, when the external electronic device (301) is included in the external electronic devices registered to the user account, the sensor (335) of the external electronic device (301) may be activated. For example, when the sensor (335) of the external electronic device (301) is activated, a user gesture may be identified to the external electronic device (301) through the activated sensor (335). For example, when at least one processor (305) transmits sensing data acquired based on the user gesture to the wearable device (100), at least one processor (300) may identify the user gesture using the sensing data. For example, at least one processor (300) may determine the external electronic device (301) from among one or more external electronic devices by identifying the user gesture using the sensing data. For example, at least one processor (300) can identify that an external electronic device (301) is registered as associated with a user account by determining the external electronic device (301) from among one or more external electronic devices.

[0117] According to another embodiment, at least one processor (305) may transmit sensing data to the server (120) via the communication circuit (325). For example, the server (120) may receive sensing data from the external electronic device (301) via the communication circuit (420). For example, the at least one processor (400) may determine the external electronic device (301) from among one or more external electronic devices by identifying a user gesture using the sensing data. For example, the at least one processor (400) may notify the wearable device (100) of the determined external electronic device (301) from among the one or more external electronic devices via the communication circuit (420).

[0118] Referring to FIG. 11B, state (1120) can be described as a state in which a sensor (335) of an external electronic device (301) including a smartwatch is activated. For example, the sensor (335) can include a gyro sensor, an acceleration sensor, and / or a barometric pressure sensor. For example, within state (1120), at least one processor (300) can display guidance (1130) on a screen to perform a user gesture related to the external electronic device (301) based on transmitting a signal to one or more external electronic devices that causes the sensors of the one or more external electronic devices to activate.

[0119] For example, the user (605) may perform a user gesture associated with the external electronic device (301) according to the guidance (1130). As a non-limiting example, the guidance (1130) may include text instructing the user to move the external electronic device (301) worn (or positioned) on the hand (1135) of the user (605) with respect to the external electronic device (301) with respect to the area (1125) on the screen, thereby allowing the user (605) to move the external electronic device (301) to the area (1125). At least one processor (300) may display a UI object representing the area (1125) on the screen in conjunction with the guidance (1130). The user gesture may include a gesture in which the user (605) moves the external electronic device (301) to be positioned within the area (1125) on the screen. When a user (605) moves an external electronic device (301) to be positioned within an area (1125) on the screen, at least one processor (300) can display on the screen an object (1140) corresponding to a hand (1115) of the user (605) moving the external electronic device (301) to be positioned within the area (1125) on the screen and a visual object (215) corresponding to the external electronic device (301). When a user (605) moves the external electronic device (301) to be positioned within the area (1125) on the screen, at least one processor (305) can identify a user gesture through a sensor (335). For example, the at least one processor (305) can identify a change in speed, acceleration, air pressure, and / or posture of the external electronic device (301) through the sensor (335), thereby identifying the user gesture. However, the present invention is not limited thereto.

[0120] For example, at least one processor (305) can obtain sensing data about a user gesture by identifying the user gesture through the sensor (335). For example, at least one processor (305) can transmit the sensing data about the user gesture to the wearable device (100) through the communication circuit (325) based on obtaining the sensing data about the user gesture. For example, at least one processor (300) can receive sensing data about the user gesture from an external electronic device (301) through the communication circuit (320). For example, at least one processor (300) can identify the user gesture using the sensing data about the user gesture.

[0121] For example, when the external electronic device (301) is included in the external electronic devices registered to the user account, the sensor (335) of the external electronic device (301) may be activated. For example, when the sensor (335) of the external electronic device (301) is activated, a user gesture may be identified to the external electronic device (301) through the activated sensor (335). For example, when at least one processor (305) transmits sensing data based on a user gesture to the wearable device (100), at least one processor (300) may identify the user gesture using the sensing data. For example, at least one processor (300) may determine the external electronic device (301) from among one or more external electronic devices by identifying the user gesture using the sensing data. For example, at least one processor (300) can identify that an external electronic device (301) is registered as associated with a user account by determining the external electronic device (301) from among one or more external electronic devices.

[0122] According to another embodiment, at least one processor (305) may transmit sensing data to the server (120) via the communication circuit (325). For example, the server (120) may receive sensing data from the external electronic device (301) via the communication circuit (420). For example, the at least one processor (400) may determine the external electronic device (301) from among one or more external electronic devices by identifying a user gesture using the sensing data. For example, the at least one processor (400) may notify the wearable device (100) of the determined external electronic device (301) from among the one or more external electronic devices via the communication circuit (420).

[0123] For example, at least one processor (300) may determine the external electronic device (301) from among one or more external electronic devices based on a visual identifier displayed on a display (345) of the external electronic device (301). Using the visual identifier displayed on the display (345) of the external electronic device (301) to determine the external electronic device (301) from among the one or more external electronic devices is exemplified within the description of FIG. 12A.

[0124] Figure 12a illustrates an example of an external electronic device determined using a visual identifier.

[0125] Referring to FIG. 12A, a state (1200) may be described as a state in which a visual identifier (1205) is displayed on a display (345) of an external electronic device (301). For example, within the state (1200), at least one processor (300) may use information about one or more external electronic devices received from a server (120) to transmit a signal to one or more external electronic devices via a communication circuit (320) to cause the one or more external electronic devices to display a visual identifier (1205). For example, one or more external electronic devices may receive a signal from the wearable device (100) via a communication circuit (e.g., a communication circuit (325) of the external electronic device (301)) to cause the display of the visual identifier (1205). For example, at least one processor (305) may display the visual identifier (1205) on the display (345) based on receiving the signal. For example, the visual identifier (1205) may include a barcode, an icon, a mark, and / or a lighting pattern of the display (345).

[0126] For example, at least one processor (300) may identify that a visual object (215) including a visual identifier (1205) is included in at least some of the images of the space in front of the wearable device (100) acquired through one or more cameras (330). For example, at least one processor (300) may display the visual object (215) including the identifier (1205) on a screen. For example, at least one processor (300) may identify that an external electronic device (301) displays the visual identifier (1205) through a display (345) based on identifying the visual object (215) including the visual identifier (1205).

[0127] For example, since the external electronic device (301) is included in the external electronic devices registered to the user account, the external electronic device (301) can display a visual identifier (1205) through the display (345). For example, at least one processor (300) can determine the external electronic device (301) from among one or more external electronic devices by identifying the visual identifier (1205) displayed through the display (345) of the external electronic device (301). For example, at least one processor (300) can identify that the external electronic device (301) is registered in connection with the user account by determining the external electronic device (301) from among one or more external electronic devices.

[0128] For example, at least one processor (300) may determine the external electronic device (301) from among one or more external electronic devices based on a software application running within the external electronic device (301). Utilizing the software application running within the external electronic device (301) to determine the external electronic device (301) from among one or more external electronic devices is exemplified within the description of FIG. 12B.

[0129] FIG. 12b illustrates an example of an external electronic device determined using a software application running within the external electronic device.

[0130] Referring to FIG. 12B, state (1210) can be described as a state in which a software application (1215) is being executed within an external electronic device (301). For example, within state (1210), at least one processor (300) can identify a software application (1215) being executed within an external electronic device (301) using information about one or more external electronic devices received from a server (120). For example, the external electronic device (301) can display a screen provided from the software application (1215) on a display (345).

[0131] For example, at least one processor (300) can identify that a visual object (215) executing a software application (1215) is included in at least some of the images of the space in front of the wearable device (100) acquired through one or more cameras (330). For example, at least one processor (300) can display the visual object (215) executing the software application (1215) on a screen. For example, at least one processor (300) can identify a software application (1215) executing in the external electronic device (301) by using a screen provided from the software application (1215) displayed through a display (345) of the external electronic device (301).

[0132] For example, at least one processor (300) may compare a software application running in an external electronic device (301) identified using information about one or more external electronic devices with a software application (1215) running in the external electronic device (301) identified using at least a portion of the images. For example, at least one processor (300) may determine the external electronic device (301) from among one or more external electronic devices based on a software application (1215) corresponding to the software application running in the external electronic device (301) identified using information about one or more external electronic devices. For example, at least one processor (300) may identify that the external electronic device (301) is registered as linked to a user account by determining the external electronic device (301) from among one or more external electronic devices.

[0133] Referring again to FIG. 7, at operation 750, at least one processor (300) may display a UI object within a screen via the display assembly (340) based on determining an external electronic device (301) from among the one or more external electronic devices using information about the one or more external electronic devices. For example, the UI object may be displayed within the screen in conjunction with a visual object corresponding to the external electronic device. The UI object displayed within the screen is exemplified in the description of FIG. 13.

[0134] Figure 13 illustrates an example of a UI object linked to a visual object.

[0135] Referring to FIG. 13, a state (1300) may be described as a state in which a UI object (1305) is displayed within a screen. For example, within the state (1300), at least one processor (300) may display a UI object (1305) within the screen based on determining an external electronic device (301) from among one or more external electronic devices registered to a user account. For example, the UI object (1305) may be associated with a visual object (215) within the screen. For example, the UI object (1305) may be available to provide a service related to the external electronic device (301) within the wearable device (100). For example, the UI object (1305) may be displayed to inquire whether to connect (or link) the wearable device (100) with the external electronic device (301).

[0136] For example, as an external electronic device (301) is registered to a user account, a user (605) may have an intention to connect (or interwork) the wearable device (100) with the external electronic device (301). For example, at least one processor (300) may display the UI object (1305) based on the external electronic device (301) registered to the user account, and may refrain from (or stop, bypass, or not display) displaying the UI object (1305) based on the external electronic device (301) not registered to the user account, thereby displaying the UI object (1305) only when the user (605) has an intention to connect (or interwork) the wearable device (100) with the external electronic device (301). For example, at least one processor (300) can display a UI object (1305) only when the user (605) intends to connect (or link) the wearable device (100) with an external electronic device (301), thereby eliminating inconvenience to the user (605) caused by displaying the UI object (1305) on the screen regardless of whether the external electronic device (301) is registered to the user account of the wearable device (200).

[0137] For example, at least one processor (300) may receive an input (1310) to a UI object (1305). For example, the input (1310) may include a touch input to the UI object (1305). For example, at least one processor (300) may identify a touch input to the UI object (1305) based on an object corresponding to a hand of a user (605).

[0138] For example, at least one processor (300) may connect (or interwork) the wearable device (100) and the external electronic device (301) based on the input (1310). For example, at least one processor (300) may connect (or interwork) with the external electronic device (301) directly (or through the server (120)). For example, at least one processor (300) may provide a service related to the external electronic device (301) within the wearable device (100) by connecting (or interworking) the wearable device (100) and the external electronic device (301). For example, services related to an external electronic device (301) may include a service for controlling a wearable device (100) using the external electronic device (301), a service for displaying the screen of the external electronic device (301) on the display assembly (340) of the wearable device (100), and / or a service for providing an event detected within the external electronic device (301) within the wearable device (100).

[0139] For example, at least one processor (300) may identify that an external electronic device (301) is not registered to a user account by not determining the external electronic device (301) among the one or more external electronic devices using information about the one or more external electronic devices. Examples of external electronic devices (301) that are not registered to a user account are illustrated within the descriptions of FIGS. 14A and 14B.

[0140] Figures 14a and 14b illustrate examples of other UI objects for external electronic devices that are not registered to a user account.

[0141] Referring to FIG. 15, a state (1400) may be described as a state in which an external electronic device (301) that is not registered to a user account is displayed. For example, within the state (1400), at least one processor (300) may use information about one or more external electronic devices to identify a distance between the wearable device (100) and one or more external electronic devices. For example, the at least one processor (300) may not determine the external electronic device (301) from among the one or more external electronic devices by failing to identify one or more external electronic devices located within a reference distance from the wearable device (100). For example, by not determining the external electronic device (301) from among the one or more external electronic devices, the external electronic device (301) may be described as an external electronic device that is not registered to a user account.

[0142] For example, at least one processor (300) may refrain from (or stop, bypass, or not display) a UI object (e.g., UI object (1305) of FIG. 13) available for providing a service related to an external electronic device (301) within the wearable device (100) (or for inquiring whether to connect (or link) the wearable device (100) and the external electronic device (301)) based on an external electronic device (301) that is not registered to a user account. For example, at least one processor (300) may display another UI object (1405) available for inquiring whether to link the external electronic device (301) with the user account within the screen based on an external electronic device (301) that is not registered to a user account. For example, the other UI object (1405) may be associated with a visual object (215) corresponding to the external electronic device (301). For example, another UI object (1405) may be displayed as a replacement for a UI object (e.g., UI object (1305) of FIG. 13).

[0143] For example, at least one processor (300) may receive an input (1410) for another UI object (1405). For example, the input (1410) may include a touch input for another UI object (1405). For example, at least one processor (300) may identify a touch input for another UI object (1405) based on an object corresponding to a hand of a user (605).

[0144] For example, at least one processor (300) may register an external electronic device (301) in association with a user account based on input (1410). For example, at least one processor (300) may connect (or link) the wearable device (100) and the external electronic device (301) by registering the external electronic device (301) in association with a user account.

[0145] As a non-limiting example, at least one processor (300) may display a UI object (e.g., UI object (1305) of FIG. 13) as a replacement for another UI object (1410) by registering an external electronic device (301) in association with a user account. At least one processor (300) may connect (or link) the wearable device (100) and the external electronic device (301) based on an input to the UI object. However, the present invention is not limited thereto.

[0146] For example, at least one processor (300) may provide a service related to the external electronic device (301) within the wearable device (100) by connecting (or interlocking) the wearable device (100) and the external electronic device (301). For example, the service related to the external electronic device (301) may include a service for controlling the wearable device (100) using the external electronic device (301), a service for displaying the screen of the external electronic device (301) on the display assembly (340) of the wearable device (100), and / or a service for providing an event detected within the external electronic device (301) within the wearable device (100).

[0147] Referring to FIG. 14B, state (1415) may be described as a state in which an external electronic device (301) that is not registered to a user account is displayed. For example, within state (1415), at least one processor (300) may use information about one or more external electronic devices to identify a distance between the wearable device (100) and one or more external electronic devices. For example, at least one processor (300) may identify one or more external electronic devices located within a reference distance from the wearable device (100). For example, among the one or more external electronic devices, an external electronic device other than the external electronic device (301) corresponding to the visual object (215) may be located within the reference distance from the wearable device (100). For example, the other external electronic device may be described as an external electronic device registered to a user account.

[0148] For example, an external electronic device (301) corresponding to a visual object (215) may not be included among one or more external electronic devices. For example, an external electronic device (301) corresponding to a visual object (215) may be described as an external electronic device that is not registered to a user account. For example, at least one processor (300) may have an error in determining which external electronic device the user intends to connect (or link) with the wearable device (100) among other external electronic devices and the external electronic device (301).

[0149] To resolve such errors, at least one processor (300) may, based on identifying an external electronic device (301) corresponding to a visual object (215) located within a reference distance from the wearable device (100) among one or more external electronic devices, display a UI object (1420) for inquiring about an external electronic device to be connected (or linked) with the wearable device (100) among the other external electronic devices and the external electronic device (301) through the display assembly (340). For example, the UI object (1420) may be displayed in association with the visual object (215).

[0150] For example, the UI object (1420) may include text for requesting selection of an external electronic device to be connected to the wearable device (100). For example, the UI object (1420) may include an executable object (1430) corresponding to an external electronic device (301) corresponding to a visual object (215) and an executable object (1425) corresponding to the other external electronic device.

[0151] For example, at least one processor (300) may receive an input (1435) for an executable object (1430) corresponding to an external electronic device (301) corresponding to a visual object (215). For example, at least one processor (300) may register the external electronic device (301) as associated with a user account based on the input (1435). For example, at least one processor (300) may connect (or link) the wearable device (100) and the external electronic device (301) by registering the external electronic device (301) as associated with a user account.

[0152] For example, at least one processor (300) may receive an input (1440) for an executable object (1425) corresponding to another external electronic device. For example, at least one processor (300) may connect (or interwork) the wearable device (100) with the other external electronic device based on the input (1440).

[0153] For example, whether an external electronic device (301) is registered as linked to a user account of a wearable device (100) can be identified by the server (120). For example, the server (120) can determine the external electronic device (301) from among one or more external electronic devices. Identifying whether the server (120) is registered as linked to a user account of a wearable device (100) is exemplified in the description of FIG. 15.

[0154] Figure 15 is a signal flow diagram between a wearable device and a server.

[0155] Referring to FIG. 15, in operation 1500, at least one processor (300) may identify whether a visual object having the shape of an electronic device (e.g., a visual object (215) of FIG. 6) is included in at least a portion of images of a space in front of a wearable device (100) acquired through one or more cameras (330). For example, operation 1500 may correspond to operation 520 of FIG. 5.

[0156] In operation 1510, at least one processor (300) may transmit information about the type of the electronic device to the server (120) via the communication circuit (320) based on identifying that a visual object having the shape of an electronic device is included in at least some of the images. At least one processor (400) may receive the information from the wearable device (100) via the communication circuit (420).

[0157] For example, at least one processor (300) may request a decision on an external electronic device corresponding to a visual object from among one or more external electronic devices by transmitting the information to the server (120) via the communication circuit (320). For example, the external electronic devices may be included in the plurality of external electronic devices (110) of FIG. 1. For example, the external electronic devices may be included in electronic devices registered in connection with a user account of the wearable device (100). For example, at least one processor (300) may identify a user account of the wearable device (100) based on unique information of a user wearing the wearable device (100) (e.g., the user's iris information or the user's password). For example, the user account may be used within the wearable device (100). For example, the user account may be related to an IoT service provided through the server (120).

[0158] In operation 1520, at least one processor (400) may identify a user account of the wearable device (100) based on receiving the information. For example, at least one processor (400) may identify external electronic devices registered to the user account of the wearable device (100).

[0159] For example, external electronic devices registered to a user account of a wearable device (100) may be registered in association with the user account of the wearable device (100) by previously being connected to the wearable device (100) or by being linked with the wearable device (100) and providing services related to the external electronic devices within the wearable device (100). For example, external electronic devices registered to a user account of a wearable device (100) may be registered in association with the user account by logging in to the user account within the external electronic devices.

[0160] For example, at least one processor (400) may identify the type of external electronic device (301) corresponding to the visual object based on receiving the request. For example, at least one processor (400) may identify one or more external electronic devices having the type of external electronic device (301) among external electronic devices registered to a user account of the wearable device (100).

[0161] In operation 1530, at least one processor (400) may identify whether the external electronic device (301) is registered as linked to a user account of the wearable device (100) based on identifying one or more external electronic devices having the type of the external electronic device (301). For example, the at least one processor (400) may determine the external electronic device (301) from among the one or more external electronic devices to identify whether the external electronic device (301) is registered as linked to a user account of the wearable device (100).

[0162] For example, at least one processor (400) can determine the external electronic device (301) from among one or more external electronic devices by controlling (or using) one or more external electronic devices. For example, at least one processor (400) can identify whether one or more external electronic devices are within a reference distance from the wearable device (100) using one or more external electronic devices. For example, at least one processor (400) can determine the external electronic device (301) from among one or more external electronic devices based on whether one or more external electronic devices are within a reference distance from the wearable device (100). Determining the external electronic device (301) from among one or more external electronic devices based on whether one or more external electronic devices are within a reference distance from the wearable device (100) may refer to the descriptions of FIGS. 9A and 9B.

[0163] For example, at least one processor (400) may identify a plurality of external electronic devices located within a reference distance from one or more external electronic devices. For example, it may be required to determine one external electronic device (301) from among a plurality of external electronic devices located within a reference distance from a wearable device (100).

[0164] For example, at least one processor (400) may determine the closest external electronic device (301) to the wearable device (100) from among a plurality of external electronic devices. For example, the description of FIG. 10 may be referred to for determining the closest external electronic device (301) to the wearable device (100) from among a plurality of external electronic devices.

[0165] For example, at least one processor (400) may use the sensor (355) of the external electronic device (301) to determine the external electronic device (301) from among one or more external electronic devices. For example, the determination of the external electronic device (301) from among one or more external electronic devices using the sensor (355) of the external electronic device (301) may refer to the descriptions of FIGS. 11A and 11B.

[0166] For example, at least one processor (400) may control the external electronic device (301) to display a visual identifier on the display (345). For example, at least one processor (400) may determine the external electronic device (301) from among one or more external electronic devices based on the visual identifier displayed on the display (345) of the external electronic device (301). Determining the external electronic device (301) from among one or more external electronic devices based on the visual identifier displayed on the display (345) of the external electronic device (301) may refer to the description of FIG. 12A.

[0167] For example, at least one processor (400) may obtain information about a software application running within an external electronic device (301). For example, at least one processor (300) may determine the external electronic device (301) from among one or more external electronic devices based on the software application running within the external electronic device (301). Determining the external electronic device (301) from among one or more external electronic devices based on the software application running within the external electronic device (301) may refer to the description of FIG. 12B.

[0168] In operation 1540, at least one processor (400) may transmit information about the decision to the wearable device (100) through the communication circuit (420) based on determining the external electronic device (301) from among one or more external electronic devices. The wearable device (100) may receive information about the decision from the server (120) through the communication circuit (320).

[0169] For example, at least one processor (300) may identify, based on information about the received decision, that an external electronic device (301) corresponding to the visual object is registered as linked to a user account.

[0170] In operation 1550, at least one processor (300) may display a UI object associated with a visual object corresponding to an external electronic device (301) stored in association with a user account via a display assembly (340). For example, displaying the UI object may refer to the description of FIG. 13.

[0171] According to various embodiments, the wearable device (100) may analyze images of the external electronic device (301) collected from one or more cameras (330), or may obtain depth or distance information (between the wearable device and the external electronic device) of the external electronic device (301) by using a depth sensor. The wearable device (100) or the server (120) may compare the acquired depth information with distance information acquired by means of communication strength information, etc., to determine the external electronic device (301) corresponding to the visual object (215). For example, if there is a large difference between distance information acquired by using information such as communication strength acquired from external electronic devices or the server (120) registered to the user account and distance information acquired by using the camera sensor of the wearable device (100), the external electronic device (301) corresponding to the visual object (215) may not be an electronic device registered to the user account.

[0172] FIG. 16 is a block diagram of an electronic device within a network environment according to various embodiments.

[0173] Referring to FIG. 16, in a network environment (1600), an electronic device (1601) may communicate with an electronic device (1602) via a first network (1698) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (1604) or a server (1608) via a second network (1699) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (1601) may communicate with the electronic device (1604) via the server (1608). According to one embodiment, the electronic device (1601) may include a processor (1620), a memory (1630), an input module (1650), an audio output module (1655), a display module (1660), an audio module (1670), a sensor module (1676), an interface (1677), a connection terminal (1678), a haptic module (1679), a camera module (1680), a power management module (1688), a battery (1689), a communication module (1690), a subscriber identification module (1696), or an antenna module (1697). In some embodiments, the electronic device (1601) may omit at least one of these components (e.g., the connection terminal (1678)), or may have one or more other components added. In some embodiments, some of these components (e.g., sensor module (1676), camera module (1680), or antenna module (1697)) may be integrated into a single component (e.g., display module (1660)).

[0174] The processor (1620) may control at least one other component (e.g., a hardware or software component) of the electronic device (1601) connected to the processor (1620) by executing, for example, software (e.g., a program (1640)), and may perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (1620) may store commands or data received from other components (e.g., a sensor module (1676) or a communication module (1690)) in a volatile memory (1632), process the commands or data stored in the volatile memory (1632), and store result data in a non-volatile memory (1634). According to one embodiment, the processor (1620) may include a main processor (1621) (e.g., a central processing unit or an application processor) or an auxiliary processor (1623) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (1621). For example, when the electronic device (1601) includes the main processor (1621) and the auxiliary processor (1623), the auxiliary processor (1623) may be configured to use less power than the main processor (1621) or to be specialized for a given function. The auxiliary processor (1623) may be implemented separately from the main processor (1621) or as a part thereof.

[0175] The auxiliary processor (1623) may control at least a portion of functions or states associated with at least one component (e.g., the display module (1660), the sensor module (1676), or the communication module (1690)) of the electronic device (1601), for example, on behalf of the main processor (1621) while the main processor (1621) is in an inactive (e.g., sleep) state, or together with the main processor (1621) while the main processor (1621) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (1623) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (1680) or a communication module (1690)). In one embodiment, the auxiliary processor (1623) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (1601) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (1608)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0176] The memory (1630) can store various data used by at least one component (e.g., the processor (1620) or the sensor module (1676)) of the electronic device (1601). The data can include, for example, software (e.g., the program (1640)) and input data or output data for commands related thereto. The memory (1630) can include volatile memory (1632) or non-volatile memory (1634).

[0177] The program (1640) may be stored as software in memory (1630) and may include, for example, an operating system (1642), middleware (1644), or an application (1646).

[0178] The input module (1650) can receive commands or data to be used in a component of the electronic device (1601) (e.g., a processor (1620)) from an external source (e.g., a user) of the electronic device (1601). The input module (1650) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0179] The audio output module (1655) can output audio signals to the outside of the electronic device (1601). The audio output module (1655) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0180] The display module (1660) can visually provide information to an external party (e.g., a user) of the electronic device (1601). The display module (1660) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. In one embodiment, the display module (1660) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0181] The audio module (1670) can convert sound into an electrical signal, or vice versa. According to one embodiment, the audio module (1670) can acquire sound through the input module (1650), output sound through the sound output module (1655), or an external electronic device (e.g., electronic device (1602)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (1601).

[0182] The sensor module (1676) can detect the operating status (e.g., power or temperature) of the electronic device (1601) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (1676) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0183] The interface (1677) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (1601) with an external electronic device (e.g., the electronic device (1602)). In one embodiment, the interface (1677) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0184] The connection terminal (1678) may include a connector through which the electronic device (1601) may be physically connected to an external electronic device (e.g., the electronic device (1602)). In one embodiment, the connection terminal (1678) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0185] The haptic module (1679) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. In one embodiment, the haptic module (1679) may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0186] The camera module (1680) can capture still images and videos. In one embodiment, the camera module (1680) may include one or more lenses, image sensors, image signal processors, or flashes.

[0187] The power management module (1688) can manage the power supplied to the electronic device (1601). According to one embodiment, the power management module (1688) can be implemented as at least a part of, for example, a power management integrated circuit (PMIC).

[0188] A battery (1689) may power at least one component of the electronic device (1601). In one embodiment, the battery (1689) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0189] The communication module (1690) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (1601) and an external electronic device (e.g., electronic device (1602), electronic device (1604), or server (1608)), and the performance of communication through the established communication channel. The communication module (1690) may operate independently from the processor (1620) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (1690) may include a wireless communication module (1692) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (1694) (e.g., a local area network (LAN) communication module, or a power line communication module). Any of these communication modules may communicate with an external electronic device (1604) via a first network (1698) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (1699) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a local area network or a wide area network)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (1692) may use subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (1696) to identify or authenticate the electronic device (1601) within a communication network such as the first network (1698) or the second network (1699).

[0190] The wireless communication module (1692) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency communications (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (1692) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (1692) can support various technologies for securing performance in high-frequency bands, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (1692) can support various requirements specified in the electronic device (1601), an external electronic device (e.g., the electronic device (1604)), or a network system (e.g., the second network (1699)). According to one embodiment, the wireless communication module (1692) may support a peak data rate (e.g., 20 Gbps or more) for eMBB implementation, a loss coverage (e.g., 164 dB or less) for mMTC implementation, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC implementation.

[0191] The antenna module (1697) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (1697) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (1697) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (1698) or the second network (1699), may be selected from the plurality of antennas by, for example, the communication module (1690). A signal or power may be transmitted or received between the communication module (1690) and the external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (1697).

[0192] According to various embodiments, the antenna module (1697) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band.

[0193] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0194] According to one embodiment, commands or data may be transmitted or received between the electronic device (1601) and an external electronic device (1604) via a server (1608) connected to a second network (1699). Each of the external electronic devices (1602 or 1604) may be the same or a different type of device as the electronic device (1601). According to one embodiment, all or part of the operations executed in the electronic device (1601) may be executed in one or more of the external electronic devices (1602, 1604, or 1608). For example, when the electronic device (1601) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (1601) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (1601). The electronic device (1601) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (1601) may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (1604) may include an Internet of Things (IoT) device. The server (1608) may be an intelligent server utilizing machine learning and / or a neural network.According to one embodiment, an external electronic device (1604) or server (1608) may be included within the second network (1699). The electronic device (1601) may be applied to intelligent services (e.g., smart homes, smart cities, smart cars, or healthcare) based on 5G communication technology and IoT-related technology.

[0195] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0196] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0197] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0198] Various embodiments of the present document may be implemented as software (e.g., a program (1640)) including one or more instructions stored in a storage medium (e.g., an internal memory (1636) or an external memory (1638)) readable by a machine (e.g., an electronic device (1601)). For example, a processor (e.g., a processor (1620)) of the machine (e.g., an electronic device (1601)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0199] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as included in a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0200] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0201] Figure 17a shows an example of a perspective view of a wearable device.

[0202] FIG. 17b illustrates an example of one or more hardware devices arranged within a wearable device.

[0203] FIG. 17A illustrates an example of a perspective view of a wearable device. FIG. 17B illustrates an example of one or more hardware components arranged within the wearable device. According to one embodiment, the wearable device (100) may have a form of glasses that can be worn on a body part (e.g., head) of a user. The wearable device (100) of FIGS. 17A and 17B may be an example of the wearable device (100) of FIG. 1. The wearable device (100) may include a head-mounted display (HMD). For example, the housing of the wearable device (100) may include a flexible material, such as rubber and / or silicone, that is configured to fit closely to a portion of the user's head (e.g., a portion of the face surrounding both eyes). For example, the housing of the wearable device (100) may include one or more straps capable of being twined around the user's head, and / or one or more temples attachable to the ears of the head.

[0204] Referring to FIG. 17A, according to one embodiment, a wearable device (100) may include at least one display (1750) and a frame (1700) supporting at least one display (1750).

[0205] According to one embodiment, a wearable device (100) may be worn on a part of a user's body. The wearable device (100) may provide augmented reality (AR), virtual reality (VR), or mixed reality (MR) that combines augmented reality and virtual reality to a user wearing the wearable device (100). For example, the wearable device (100) may display a virtual reality image provided from at least one optical device (1782, 1784) of FIG. 17B on at least one display (1750) in response to a user's designated gesture acquired through the motion recognition cameras (1760-2, 1760-3) of FIG. 17B.

[0206] According to one embodiment, at least one display (1750) may provide visual information to a user. For example, at least one display (1750) may include a transparent or translucent lens. At least one display (1750) may include a first display (1750-1) and / or a second display (1750-2) spaced apart from the first display (1750-1). For example, the first display (1750-1) and the second display (1750-2) may be positioned at positions corresponding to the user's left and right eyes, respectively.

[0207] Referring to FIG. 17B, at least one display (1750) can provide the user with visual information transmitted from external light and other visual information distinct from the visual information through a lens included in the at least one display (1750). The lens can be formed based on at least one of a Fresnel lens, a pancake lens, or a multi-channel lens. For example, the at least one display (1750) can include a first surface (1731) and a second surface (1732) opposite the first surface (1731). A display area can be formed on the second surface (1732) of the at least one display (1750). When the user wears the wearable device (100), external light can be transmitted to the user by being incident on the first surface (1731) and transmitted through the second surface (1732). As another example, at least one display (1750) can display an augmented reality image combined with a virtual reality image provided from at least one optical device (1782, 1784) on a real screen transmitted through external light, in a display area formed on the second surface (1732).

[0208] In one embodiment, at least one display (1750) may include at least one waveguide (1733, 1734) that diffracts light emitted from at least one optical device (1782, 1784) and transmits the diffracted light to a user. The at least one waveguide (1733, 1734) may be formed based on at least one of glass, plastic, or polymer. A nanopattern may be formed on at least a portion of the exterior or interior of the at least one waveguide (1733, 1734). The nanopattern may be formed based on a grating structure having a polygonal and / or curved shape. Light incident on one end of the at least one waveguide (1733, 1734) may be propagated to the other end of the at least one waveguide (1733, 1734) by the nanopattern. At least one waveguide (1733, 1734) may include at least one diffractive element (e.g., a diffractive optical element (DOE), a holographic optical element (HOE)) and at least one reflective element (e.g., a reflective mirror). For example, at least one waveguide (1733, 1734) may be arranged within the wearable device (100) to guide a screen displayed by at least one display (1750) to the user's eyes. For example, the screen may be transmitted to the user's eyes based on total internal reflection (TIR) ​​occurring within the at least one waveguide (1733, 1734).

[0209] The wearable device (100) can analyze an object included in a real image collected through a shooting camera (1760-4), combine a virtual object corresponding to an object to be provided with augmented reality among the analyzed objects, and display the virtual object on at least one display (1750). The virtual object can include at least one of text and an image regarding various information related to the object included in the real image. The wearable device (100) can analyze the object based on a multi-camera such as a stereo camera. For the object analysis, the wearable device (100) can perform spatial recognition (e.g., simultaneous localization and mapping (SLAM)) using a multi-camera and / or time-of-flight (ToF). A user wearing the wearable device (100) can view an image displayed on at least one display (1750).

[0210] According to one embodiment, the frame (1700) may be formed as a physical structure that allows the wearable device (100) to be worn on the user's body. According to one embodiment, the frame (1700) may be configured so that, when the user wears the wearable device (100), the first display (1750-1) and the second display (1750-2) can be positioned corresponding to the user's left and right eyes. The frame (1700) may support at least one display (1750). For example, the frame (1700) may support the first display (1750-1) and the second display (1750-2) to be positioned corresponding to the user's left and right eyes.

[0211] Referring to FIG. 17A, the frame (1700) may include a region (1720) that at least partially contacts a portion of the user's body when the user wears the wearable device (100). For example, the region (1720) of the frame (1700) that contacts a portion of the user's body may include a region that contacts a portion of the user's nose, a portion of the user's ear, and a portion of the side of the user's face that the wearable device (100) makes contact with. According to one embodiment, the frame (1700) may include a nose pad (1710) that contacts a portion of the user's body. When the wearable device (100) is worn by the user, the nose pad (1710) may contact a portion of the user's nose. The frame (1700) may include a first temple (1704) and a second temple (1705) that contact another part of the user's body that is distinct from the part of the user's body.

[0212] For example, the frame (1700) may include a first rim (1701) that surrounds at least a portion of the first display (1750-1), a second rim (1702) that surrounds at least a portion of the second display (1750-2), a bridge (1703) that is disposed between the first rim (1701) and the second rim (1702), a first pad (1711) that is disposed along a portion of the edge of the first rim (1701) from one end of the bridge (1703), a second pad (1712) that is disposed along a portion of the edge of the second rim (1702) from the other end of the bridge (1703), a first temple (1704) that extends from the first rim (1701) and is fixed to a portion of the ear of the wearer, and a second temple (1705) that extends from the second rim (1702) and is fixed to a portion of the ear opposite the ear. There are. The first pad (1711) and the second pad (1712) can be in contact with a part of the user's nose, and the first temple (1704) and the second temple (1705) can be in contact with a part of the user's face and a part of the user's ear. The temples (1704, 1705) can be rotatably connected to the rim through the hinge units (1706, 1707) of FIG. 17B. The first temple (1704) can be rotatably connected to the first rim (1701) through the first hinge unit (1706) disposed between the first rim (1701) and the first temple (1704). The second temple (1705) may be rotatably connected to the second rim (1702) via a second hinge unit (1707) disposed between the second rim (1702) and the second temple (1705). In one embodiment, the wearable device (100) may use a touch sensor, a grip sensor, and / or a proximity sensor formed on at least a portion of a surface of the frame (1700) to identify an external object (e.g., a user's fingertip) touching the frame (1700) and / or a gesture performed by the external object.

[0213] According to one embodiment, the wearable device (100) may include hardwares that perform various functions (e.g., hardwares to be described later based on the block diagram of FIG. 4). For example, the hardwares may include a battery module (1770), an antenna module (1775), at least one optical device (1782, 1784), speakers (e.g., speakers 1755-1, 1755-2), a microphone (e.g., microphones 1765-1, 1765-2, 1765-3), a light-emitting module (not shown), and / or a printed circuit board (PCB) (1790) (e.g., a printed circuit board). The various hardwares may be arranged within a frame (1700).

[0214] According to one embodiment, microphones (e.g., microphones 1765-1, 1765-2, 1765-3) of the wearable device (100) may be disposed on at least a portion of the frame (1700) to acquire sound signals. A first microphone (1765-1) disposed on the bridge (1703), a second microphone (1765-2) disposed on the second rim (1702), and a third microphone (1765-3) disposed on the first rim (1701) are illustrated in FIG. 17B , but the number and arrangement of the microphones (1765) are not limited to the embodiment of FIG. 17B . When the number of microphones (1765) included in the wearable device (100) is two or more, the wearable device (100) can identify the direction of a sound signal by using a plurality of microphones placed on different parts of the frame (1700).

[0215] In one embodiment, at least one optical device (1782, 1784) may project a virtual object onto at least one display (1750) to provide various image information to a user. For example, at least one optical device (1782, 1784) may be a projector. At least one optical device (1782, 1784) may be disposed adjacent to at least one display (1750) or may be included within at least one display (1750) as a part of at least one display (1750). In one embodiment, the wearable device (100) may include a first optical device (1782) corresponding to a first display (1750-1) and a second optical device (1784) corresponding to a second display (1750-2). For example, at least one optical device (1782, 1784) may include a first optical device (1782) positioned at an edge of a first display (1750-1) and a second optical device (1784) positioned at an edge of a second display (1750-2). The first optical device (1782) may transmit light to a first waveguide (1733) positioned on the first display (1750-1), and the second optical device (1784) may transmit light to a second waveguide (1734) positioned on the second display (1750-2).

[0216] In one embodiment, the camera (1760) may include a recording camera (1760-4), an eye tracking camera (ET CAM) (1760-1), and / or a motion recognition camera (1760-2, 1760-3). The recording camera (1760-4), the eye tracking camera (1760-1), and the motion recognition cameras (1760-2, 1760-3) may be positioned at different locations on the frame (1700) and may perform different functions. The eye tracking camera (1760-1) may output data indicating the position or gaze of the eyes of a user wearing the wearable device (100). For example, the wearable device (100) may detect the gaze from an image including the user's pupils obtained through the eye tracking camera (1760-1). The wearable device (100) can identify an object (e.g., a real object and / or a virtual object) focused on by the user using the user's gaze acquired through the gaze tracking camera (1760-1). The wearable device (100) that has identified the focused object can execute a function (e.g., gaze interaction) for interaction between the user and the focused object. The wearable device (100) can express a part corresponding to the eye of an avatar representing the user in a virtual space using the user's gaze acquired through the gaze tracking camera (1760-1). The wearable device (100) can render an image (or screen) displayed on at least one display (1750) based on the position of the user's eyes. For example, the visual quality of a first region related to the gaze within the image and the visual quality (e.g., resolution, brightness, saturation, grayscale, PPI) of a second region distinguished from the first region may be different from each other.The wearable device (100) can obtain an image having a visual quality of a first area matching the user's gaze and a visual quality of a second area using foveated rendering. For example, if the wearable device (100) supports an iris recognition function, user authentication can be performed based on iris information obtained using a gaze tracking camera (1760-1). An example in which the gaze tracking camera (1760-1) is positioned toward the user's right eye is illustrated in FIG. 17B, but the embodiment is not limited thereto, and the gaze tracking camera (1760-1) can be positioned solely toward the user's left eye, or toward both eyes.

[0217] In one embodiment, the capturing camera (1760-4) can capture an actual image or background to be aligned with a virtual image to implement augmented reality or mixed reality content. The capturing camera (1760-4) can be used to obtain a high-resolution image based on HR (high resolution) or PV (photo video). The capturing camera (1760-4) can capture an image of a specific object existing at a location viewed by the user and provide the image to at least one display (1750). The at least one display (1750) can display a single image in which information about an actual image or background including an image of the specific object obtained using the capturing camera (1760-4) and a virtual image provided through at least one optical device (1782, 1784) are superimposed. The wearable device (100) can compensate for depth information (e.g., the distance between the wearable device (100) and an external object acquired through a depth sensor) using an image acquired through the capture camera (1760-4). The wearable device (100) can perform object recognition using an image acquired using the capture camera (1760-4). The wearable device (100) can perform a function of focusing on an object (or subject) in an image (e.g., auto focus) and / or an optical image stabilization (OIS) function (e.g., anti-shake function) using the capture camera (1760-4). The wearable device (100) can perform a pass-through function to display an image acquired through the capture camera (1760-4) by overlapping at least a portion of a screen representing a virtual space on at least one display (1750) while displaying the screen. In one embodiment, the camera (1760-4) may be positioned on a bridge (1703) positioned between the first rim (1701) and the second rim (1702).

[0218] The gaze tracking camera (1760-1) can implement more realistic augmented reality by tracking the gaze of a user wearing the wearable device (100) and matching the user's gaze with visual information provided to at least one display (1750). For example, when the wearable device (100) looks straight ahead, the wearable device (100) can naturally display environmental information related to the user's front at a location where the user is located on at least one display (1750). The gaze tracking camera (1760-1) can be configured to capture an image of the user's pupil to determine the user's gaze. For example, the gaze tracking camera (1760-1) can receive gaze detection light reflected from the user's pupil and track the user's gaze based on the position and movement of the received gaze detection light. In one embodiment, the gaze tracking camera (1760-1) can be positioned at positions corresponding to the user's left and right eyes. For example, the gaze tracking camera (1760-1) may be positioned within the first rim (1701) and / or the second rim (1702) to face the direction in which the user wearing the wearable device (100) is positioned.

[0219] The gesture recognition cameras (1760-2, 1760-3) can recognize the movement of the user's entire body, such as the user's torso, hands, or face, or a part of the body, and thereby provide a specific event on a screen provided on at least one display (1750). The gesture recognition cameras (1760-2, 1760-3) can recognize the user's gesture (gesture recognition), obtain a signal corresponding to the gesture, and provide a display corresponding to the signal on at least one display (1750). The processor can identify the signal corresponding to the gesture, and perform a designated function based on the identification. The gesture recognition cameras (1760-2, 1760-3) can be used to perform a spatial recognition function using SLAM and / or a depth map for 6 degrees of freedom pose (6 dof pose). The processor may perform gesture recognition and / or object tracking functions using the motion recognition cameras (1760-2, 1760-3). In one embodiment, the motion recognition cameras (1760-2, 1760-3) may be positioned on the first rim (1701) and / or the second rim (1702).

[0220] The camera (1760) included in the wearable device (100) is not limited to the above-described gaze tracking camera (1760-1) and motion recognition cameras (1760-2, 1760-3). For example, the wearable device (100) can identify an external object included in the FoV using a camera positioned toward the user's FoV. The wearable device (100) can identify an external object based on a sensor for identifying the distance between the wearable device (100) and the external object, such as a depth sensor and / or a time of flight (ToF) sensor. The camera (1760) positioned toward the FoV can support an autofocus function and / or an optical image stabilization (OIS) function. For example, the wearable device (100) may include a camera (1760) (e.g., a face tracking (FT) camera) positioned toward the face to obtain an image including the face of a user wearing the wearable device (100).

[0221] Although not shown, in one embodiment, the wearable device (100) may further include a light source (e.g., an LED) that emits light toward a subject (e.g., a user's eyes, face, and / or an external object within the FoV) being captured using the camera (1760). The light source may include an infrared wavelength LED. The light source may be disposed on at least one of the frame (1700) and the hinge units (1706, 1707).

[0222] According to one embodiment, the battery module (1770) may supply power to the electronic components of the wearable device (100). In one embodiment, the battery module (1770) may be disposed within the first temple (1704) and / or the second temple (1705). For example, the battery module (1770) may be a plurality of battery modules (1770). The plurality of battery modules (1770) may be disposed within each of the first temple (1704) and the second temple (1705). In one embodiment, the battery module (1770) may be disposed at an end of the first temple (1704) and / or the second temple (1705).

[0223] The antenna module (1775) can transmit signals or power to the outside of the wearable device (100), or receive signals or power from the outside. In one embodiment, the antenna module (1775) can be positioned within the first temple (1704) and / or the second temple (1705). For example, the antenna module (1775) can be positioned close to one surface of the first temple (1704) and / or the second temple (1705).

[0224] The speaker (1755) can output an audio signal to the outside of the wearable device (100). The audio output module may be referred to as a speaker. In one embodiment, the speaker (1755) may be positioned within the first temple (1704) and / or the second temple (1705) so as to be positioned adjacent to the ear of a user wearing the wearable device (100). For example, the speaker (1755) may include a second speaker (1755-2) positioned within the first temple (1704) and thus adjacent to the user's left ear, and a first speaker (1755-1) positioned within the second temple (1705) and thus adjacent to the user's right ear.

[0225] The light-emitting module (not shown) may include at least one light-emitting element. The light-emitting module may emit light of a color corresponding to a specific state or emit light with an action corresponding to a specific state in order to visually provide information regarding a specific state of the wearable device (100) to the user. For example, when the wearable device (100) requires charging, it may emit red light at a regular cycle. In one embodiment, the light-emitting module may be disposed on the first rim (1701) and / or the second rim (1702).

[0226] Referring to FIG. 17B, according to one embodiment, a wearable device (100) may include a printed circuit board (PCB) (1790). The PCB (1790) may be included in at least one of the first temple (1704) or the second temple (1705). The PCB (1790) may include an interposer disposed between at least two sub-PCBs. One or more hardwares included in the wearable device (100) (e.g., hardwares illustrated by different blocks in FIG. 4) may be disposed on the PCB (1790). The wearable device (100) may include a flexible PCB (FPCB) for interconnecting the hardwares.

[0227] According to one embodiment, a wearable device (100) may include at least one of a gyro sensor, a gravity sensor, and / or an acceleration sensor for detecting a posture of the wearable device (100) and / or a posture of a body part (e.g., a head) of a user wearing the wearable device (100). Each of the gravity sensor and the acceleration sensor may measure gravitational acceleration and / or acceleration based on mutually perpendicular designated three-dimensional axes (e.g., an x-axis, a y-axis, and a z-axis). The gyro sensor may measure an angular velocity of each of the designated three-dimensional axes (e.g., an x-axis, a y-axis, and a z-axis). At least one of the gravity sensor, the acceleration sensor, and the gyro sensor may be referred to as an inertial measurement unit (IMU). According to one embodiment, the wearable device (100) may identify a user's motion and / or gesture performed to execute or terminate a specific function of the wearable device (100) based on the IMU.

[0228] Figures 18a and 18b show an example of the appearance of a wearable device.

[0229] FIGS. 18A and 18B illustrate an example of an exterior appearance of a wearable device (e.g., a wearable device (100)). The wearable device (100) of FIGS. 18A and 18B may be an example of the wearable device (100) of FIG. 1. According to one embodiment, an example of an exterior appearance of a first side (1810) of a housing of the wearable device (100) may be illustrated in FIG. 18A, and an example of an exterior appearance of a second side (1820) opposite to the first side (1810) may be illustrated in FIG. 18B.

[0230] Referring to FIG. 18A, according to one embodiment, a first surface (1810) of a wearable device (100) may have a form attachable to a body part of a user (e.g., the face of the user). Although not shown, the wearable device (100) may further include a strap for fixing to a body part of a user, and / or one or more temples (e.g., the first temple (1704) and / or the second temple (1705) of FIGS. 17A and 17B). A first display (1750-1) for outputting an image to a left eye among the user's two eyes, and a second display (1750-2) for outputting an image to a right eye among the two eyes, may be disposed on the first surface (1810). The wearable device (100) may be formed on the first surface (1810) and may further include a rubber or silicone packing to prevent interference from light (e.g., ambient light) different from the light emitted from the first display (1750-1) and the second display (1750-2).

[0231] According to one embodiment, the wearable device (100) may include cameras (1760-1) for photographing and / or tracking both eyes of a user adjacent to each of the first display (1750-1) and the second display (1750-2). The cameras (1760-1) may be referred to as the gaze tracking camera (1760-1) of FIG. 17B. According to one embodiment, the wearable device (100) may include cameras (1760-5, 1760-6) for photographing and / or recognizing a face of a user. The cameras (1760-5, 1760-6) may be referred to as FT cameras. The wearable device (100) may control an avatar representing the user in a virtual space based on the motion of the user's face identified using the cameras (1760-5, 1760-6). For example, the wearable device (100) may change the texture and / or shape of a portion of an avatar (e.g., a portion of an avatar representing a human face) using information obtained by cameras (1760-5, 1760-6) (e.g., FT cameras) and representing a facial expression of a user wearing the wearable device (100).

[0232] Referring to FIG. 18B, a camera (e.g., cameras 1760-7, 1760-8, 1760-9, 1760-10, 1760-11, 1760-12)) and / or a sensor (e.g., a depth sensor 1830) may be disposed on a second surface (1820) opposite to the first surface (1810) of FIG. 18A to obtain information related to the external environment of the wearable device (100). For example, the cameras (1760-7, 1760-8, 1760-9, 1760-10) may be disposed on the second surface (1820) to recognize external objects. Cameras (1760-7, 1760-8, 1760-9, 1760-10) may be referenced to the motion recognition cameras (1760-2, 1760-3) of FIG. 17b.

[0233] For example, using cameras (1760-11, 1760-12), the wearable device (100) can obtain images and / or videos to be transmitted to each of the user's eyes. The camera (1760-11) can be placed on the second face (1820) of the wearable device (100) to obtain an image to be displayed through the second display (1750-2) corresponding to the right eye among the two eyes. The camera (1760-12) can be placed on the second face (1820) of the wearable device (100) to obtain an image to be displayed through the first display (1750-1) corresponding to the left eye among the two eyes. The cameras (1760-11, 1760-12) can be referred to as the shooting camera (1760-4) of FIG. 17B.

[0234] According to one embodiment, the wearable device (100) may include a depth sensor (1830) disposed on the second face (1820) to identify a distance between the wearable device (100) and an external object. Using the depth sensor (1830), the wearable device (100) may obtain spatial information (e.g., a depth map) for at least a portion of the FoV of a user wearing the wearable device (100). Although not illustrated, a microphone may be disposed on the second face (1820) of the wearable device (100) to obtain a sound output from an external object. The number of microphones may be one or more depending on the embodiment.

[0235] Hereinafter, with reference to FIG. 19, the hardware or software configuration of the wearable device (100) is described.

[0236] Figure 19 shows an example of a block diagram of a wearable device.

[0237] Fig. 19 illustrates an example of a block diagram of a wearable device (e.g., a wearable device (100)). The wearable device (100) of Fig. 19 may be an example of the wearable device (100) of Fig. 1 and the wearable devices (100) of Figs. 17a to 18b.

[0238] Referring to FIG. 19, a wearable device (100) according to one embodiment may include a processor (1910), a memory (1915), a display (1750) (e.g., the first display (1750-1) and / or the second display (1750-2) of FIGS. 17A, 17B, 18A, and 18B), and / or a sensor (1920). The processor (1910), the memory (1915), the display (1750), and / or the sensor (1920) may be electrically and / or operatively connected to each other by electronic components such as a communication bus (1902). In the present disclosure, the operative connection of the electronic components may include a direct connection established between the electronic components and / or an indirect connection established between the electronic components such that a first electronic component among the electronic components is controlled by a second electronic component among the electronic components. The type and / or number of electronic components included in the wearable device (100) is not limited to those illustrated in FIG. 19. For example, the wearable device (100) may include only some of the electronic components illustrated in FIG. 19.

[0239] A processor (1910) of a wearable device (100) according to one embodiment may include a circuit (e.g., a processing circuit) for processing data based on one or more instructions. The circuit for processing data may include, for example, an arithmetic and logic unit (ALU), a field programmable gate array (FPGA), a central processing unit (CPU), and / or an application processor (AP). In one embodiment, the wearable device (100) may include one or more processors. The processor (1910) may have a multi-core processor structure such as a dual core, a quad core, a hexa core, and / or an octa core. The multi-core processor structure of the processor (1910) may include a structure (e.g., a big-little structure) based on a plurality of core circuits that are distinguished by power consumption, clock, and / or calculation amount per unit time. In one embodiment comprising a processor (1910) having a multi-core processor architecture, the operations and / or functions of the present disclosure may be performed individually or collectively by one or more cores included in the processor (1910).

[0240] A memory (1915) of a wearable device (100) according to one embodiment may include electronic components for storing data and / or instructions input to and / or output from a processor (1910). The memory (1915) may include, for example, volatile memory such as random-access memory (RAM) and / or non-volatile memory such as read-only memory (ROM). The volatile memory may include, for example, at least one of dynamic RAM (DRAM), static RAM (SRAM), cache RAM, and pseudo SRAM (PSRAM). The non-volatile memory may include, for example, at least one of programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, hard disk, compact disc, and embedded multimedia card (eMMC). In one embodiment, memory (1915) may be referred to as storage.

[0241] In one embodiment, a display (1750) of a wearable device (100) can output visualized information to a user of the wearable device (100). The display (1750), which is arranged in front of the eyes of a user wearing the wearable device (100), can be arranged on at least a portion of a housing of the wearable device (100) (e.g., the first display (1750-1) and / or the second display (1750-2) of FIGS. 17A, 17B, 18A, and 18B). For example, the display (1750) can be controlled by a processor (1910) including circuits such as a CPU, a GPU (graphics processing unit), and / or a DPU (display processing unit), to output visualized information to the user. The display (1750) can include a flexible display, a flat panel display (FPD), and / or electronic paper. The display (1750) may include a liquid crystal display (LCD), a plasma display panel (PDP), and / or one or more light emitting diodes (LEDs). The LEDs may include organic LEDs (OLEDs). The embodiment is not limited thereto, and for example, if the wearable device (100) includes a lens for transmitting external light (or ambient light), the display (1750) may include a projector (or projection assembly) for projecting light onto the lens. In one embodiment, the display (1750) may be referred to as a display panel and / or a display module. The pixels included in the display (1750) may be arranged to face one of the user's eyes when the wearable device (100) is worn by the user.For example, the display (1750) may include display areas (or active areas) corresponding to each of the user's two eyes.

[0242] In one embodiment, a sensor (1920) of a wearable device (100) may generate electrical information that may be processed by a processor (1910) and / or a memory (1915) from non-electronic information related to the wearable device (100). For example, the sensor (1920) may include a global positioning system (GPS) sensor for detecting a geographic location of the wearable device (100). In addition to the GPS method, the sensor (1920) may generate information indicating a geographic location of the wearable device (100) based on a global navigation satellite system (GNSS), such as, for example, Galileo or Beidou (compass). The above information may be stored in memory (1915), processed by processor (1910), and / or transmitted to another electronic device distinct from the wearable device (100) via communication circuitry.

[0243] According to one embodiment, one or more instructions (or commands) representing data to be processed, calculations to be performed, and / or operations to be performed by the processor (1910) of the wearable device (100) may be stored in the memory (1915) of the wearable device (100). A set of one or more instructions may be referred to as a program, firmware, an operating system, a process, a routine, a sub-routine, and / or a software application (hereinafter, “application”). For example, the wearable device (100) and / or the processor (1910) may perform at least one of the operations of FIGS. 5 to 15 when a set of a plurality of instructions distributed in the form of an operating system, firmware, a driver, a program, and / or a software application is executed. Hereinafter, the fact that a software application is installed in a wearable device (100) may mean that one or more instructions provided in the form of a software application (or package) are stored in a memory (1915), and that the one or more applications are stored in a format executable by the processor (1910) (e.g., a file having an extension specified by the operating system of the wearable device (100)). As an example, the application may include a program and / or a library related to a service provided to a user.

[0244] Referring to FIG. 19, programs installed in the wearable device (100) may be included in any one of different layers, including an application layer (1940), a framework layer (1950), and / or a hardware abstraction layer (HAL) (1980), based on the target. For example, programs (e.g., modules or drivers) designed to target the hardware (e.g., the display (1750), and / or the sensor (1920)) of the wearable device (100) may be included in the hardware abstraction layer (1980). The framework layer (1950) may be referred to as an XR framework layer from the perspective of including one or more programs for providing an XR (extended reality) service. For example, the layers illustrated in FIG. 19 may be logically (or for convenience of explanation) separated, and may not mean that the address space of the memory (1915) is separated by the layers.

[0245] For example, within the framework layer (1950), programs designed to target at least one of the hardware abstraction layer (1980) and / or the application layer (1940) (e.g., a position tracker (1971), a space recognizer (1972), a gesture tracker (1973), an eye-gaze tracker (1974), and / or a face tracker (1975)) may be included. The programs included in the framework layer (1950) may provide an application programming interface (API) that is executable (or callable) based on other programs.

[0246] For example, the application layer (1940) may include a program designed to target users of the wearable device (100). As an example of programs included in the application layer (1940), an extended reality (XR) system user interface (UI) (1941) and / or an XR application (1942) are exemplified, but the embodiment is not limited thereto. For example, programs (e.g., software applications) included in the application layer (1940) may call an API to cause execution of functions supported by programs included in the framework layer (1950).

[0247] For example, the wearable device (100) may display one or more visual objects on the display (1750) for performing interaction with the user based on the execution of the XR system UI (1941). A visual object may refer to an object that can be placed within a screen for transmitting and / or interacting with information, such as text, an image, an icon, a video, a button, a checkbox, a radio button, a text box, a slider, and / or a table. A visual object may be referred to as a visual guide, a virtual object, a visual element, a UI element, a view object, and / or a view element. The wearable device (100) may provide the user with functions available within a virtual space based on the execution of the XR system UI (1941).

[0248] Referring to FIG. 19, a lightweight renderer (1943) and / or an XR plug-in (1944) is illustrated, but is not limited to, included within the XR system UI (1941). For example, based on the XR system UI (1941), the processor (1910) may execute a lightweight renderer (1943) and / or an XR plug-in (1944) within the framework layer (1950).

[0249] For example, the wearable device (100) may acquire resources (e.g., APIs, system processes, and / or libraries) used to define, create, and / or execute a rendering pipeline that allows partial changes based on the execution of a lightweight renderer (1943). The lightweight renderer (1943) may be referred to as a lightweight render pipeline in terms of defining a rendering pipeline that allows partial changes. The lightweight renderer (1943) may include a renderer built prior to the execution of a software application (e.g., a prebuilt renderer). For example, the wearable device (100) may acquire resources (e.g., APIs, system processes, and / or libraries) used to define, create, and / or execute an entire rendering pipeline based on the execution of an XR plug-in (1944). The XR plugin (1944) can be referred to as an open XR native client from the perspective of defining (or configuring) the entire rendering pipeline.

[0250] For example, the wearable device (100) may display a screen representing at least a portion of a virtual space on the display (1750) based on the execution of the XR application (1942). The XR plug-in (1944-1) included in the XR application (1942) may include instructions that support functions similar to those of the XR plug-in (1944) of the XR system UI (1941). Descriptions of the XR plug-in (1944-1) that overlap with those of the XR plug-in (1944) may be omitted. The wearable device (100) may cause the execution of the virtual space manager (1951) based on the execution of the XR application (1942).

[0251] For example, the wearable device (100) may display an image on the display (1750) in a virtual space based on the execution of the application (1945). The application (1945) may be configured to output image information for displaying a two-dimensional image. The wearable device (100) may cause the execution of the virtual space manager (1951) based on the execution of the application (1945). The wearable device (100) may generate dual image information to display the two-dimensional image in a three-dimensional virtual space based on the execution of the application (1945). Here, the dual image information may include first image information for the left eye and second image information for the right eye in consideration of binocular disparity. In order to display the two-dimensional image in the three-dimensional virtual space, the wearable device (100) may generate the dual image information based on the image information for displaying the two-dimensional image.

[0252] According to one embodiment, the wearable device (100) may provide a virtual space service based on the execution of the virtual space manager (1951). For example, the virtual space manager (1951) may include a platform for supporting the virtual space service. Based on the execution of the virtual space manager (1951), the wearable device (100) may identify a virtual space formed based on the user's location indicated by data acquired through the sensor (1930), and may display at least a portion of the virtual space on the display (1750). The virtual space manager (1951) may be referred to as a composition presentation manager (CPM).

[0253] For example, the virtual space manager (1951) may include a runtime service (1952). As an example, the runtime service (1952) may be referred to as an OpenXR runtime module (or an OpenXR runtime program). The wearable device (100) may execute at least one of a user's pose prediction function, a frame timing function, and / or a spatial input function based on the execution of the runtime service (1952). As an example, the wearable device (100) may perform rendering for a virtual space service to the user based on the execution of the runtime service (1952). For example, a function related to a virtual space, executable by the application layer (1940), may be supported based on the execution of the runtime service (1952).

[0254] For example, the virtual space manager (1951) may include a pass-through manager (1953). Based on the execution of the pass-through manager (1953), the wearable device (100) may display an image and / or video representing an actual space acquired through an external camera on at least a portion of the screen while displaying a screen representing a virtual space on the display (1750).

[0255] For example, the virtual space manager (1951) may include an input manager (1954). The wearable device (100) may identify data (e.g., sensor data) acquired by executing one or more programs included in the recognition service layer (1970) based on the execution of the input manager (1954). The wearable device (100) may use the acquired data to identify user input related to the wearable device (100). The user input may be related to a motion (e.g., a hand gesture), gaze, and / or speech of the user identified by a sensor (1920) (e.g., an image sensor (1930) such as an external camera). The user input may be identified based on an external electronic device connected (or paired) via a communication circuit.

[0256] For example, the perception abstract layer (1960) can be used for data exchange between the virtual space manager (1951) and the perception service layer (1970). From the perspective of being used for data exchange between the virtual space manager (1951) and the perception service layer (1970), the perception abstract layer (1960) can be referred to as an interface. For example, the perception abstract layer (1960) can be referenced as OpenPX. The perception abstract layer (1960) can be used for a perception client and a perception service.

[0257] According to one embodiment, the recognition service layer (1970) may include one or more programs for processing data acquired from the sensor (1920). The one or more programs may include at least one of a position tracker (1971), a spatial recognizer (1972), a gesture tracker (1973), and / or an eye tracker (1974). The type and / or number of the one or more programs included in the recognition service layer (1970) are not limited to those illustrated in FIG. 19.

[0258] For example, the wearable device (100) can identify the pose of the wearable device (100) using the sensor (1930) based on the execution of the position tracker (1971). The wearable device (100) can identify the 6 degrees of freedom pose (6 dof pose) of the wearable device (100) using data acquired using an external camera (e.g., an image sensor (1921)) and / or an IMU (e.g., a motion sensor (1922) including a gyro sensor, an acceleration sensor, and / or a geomagnetic sensor) based on the execution of the position tracker (1971). The position tracker (1971) may be referred to as a head tracking (HeT) module (or head tracker, head tracking program).

[0259] For example, the wearable device (100) may obtain information for providing a three-dimensional virtual space corresponding to the surrounding environment (e.g., external space) of the wearable device (100) (or the user of the wearable device (100)) based on the execution of the space recognizer (1972). The wearable device (100) may reproduce the surrounding environment of the wearable device (100) in three dimensions using data obtained using an external camera (e.g., an image sensor (1921)) based on the execution of the space recognizer (1972). The wearable device (100) may identify at least one of a plane, a slope, and stairs based on the surrounding environment of the wearable device (100) reproduced in three dimensions based on the execution of the space recognizer (1972). The space recognizer (1972) may be referred to as a scene understanding (SU) module (or a scene recognition program).

[0260] For example, the wearable device (100) may identify (or recognize) a pose and / or gesture of a hand of a user of the wearable device (100) based on the execution of the gesture tracker (1973). As an example, the wearable device (100) may identify a pose and / or gesture of a hand of a user using data acquired from an external camera (e.g., an image sensor (1921)) based on the execution of the gesture tracker (1973). As an example, the wearable device (100) may identify a pose and / or gesture of a hand of a user based on data (or images) acquired using an external camera based on the execution of the gesture tracker (1973). The gesture tracker (1973) may be referred to as a hand tracking (HaT) module (or hand tracking program) and / or a gesture tracking module.

[0261] For example, the wearable device (100) may identify (or track) eye movements of a user of the wearable device (100) based on the execution of the gaze tracker (1974). As an example, the wearable device (100) may identify eye movements of the user using data acquired from a gaze tracking camera (e.g., an image sensor (1921)) based on the execution of the gaze tracker (1974). The gaze tracker (1974) may be referred to as an eye tracking (ET) module (or eye tracking program) and / or a gaze tracking module.

[0262] For example, the recognition service layer (1970) of the wearable device (100) may further include a face tracker (1975) for tracking the user's face. For example, the wearable device (100) may identify (or track) the movement of the user's face and / or the user's expression based on the execution of the face tracker (1975). The wearable device (100) may estimate the user's expression based on the movement of the user's face based on the execution of the face tracker (1975). As an example, the wearable device (100) may identify the movement of the user's face and / or the user's expression based on data (e.g., images and / or videos) acquired using a camera (1925) (e.g., a camera directed at at least a portion of the user's face) based on the execution of the face tracker (1975).

[0263] Referring to FIG. 19, the renderer (1990) may include instructions for rendering images in a three-dimensional virtual space. The processor (1910) executing the renderer (1990) may obtain at least one image to be at least partially displayed in the display area of ​​the display (1750) in a software application. For example, the processor (1910) executing the renderer (1990) may determine the location of the area in which an application (e.g., XR application (1942), application (1945)) is to be rendered. The processor (1910) executing the renderer (1990) may generate an image of the application to be displayed on the display (1750). The renderer (1990) may synthesize images to generate a composite image to be displayed on the display (1750).

[0264] For example, the processor (1910) executing the renderer (1990) can divide the display area of ​​the display (1750) into a foveated portion (or may be referred to as the foveated area) and a peripheral portion (or may be referred to as the residual area) using the gaze position calculated using the position tracker (1971) and / or the gaze tracker (1974). For example, the processor (1910) detecting the coordinate values ​​of the gaze position can determine the portion of the display area including the coordinate values ​​as the foveated area. The DPU executing the renderer (1990) can obtain at least one image corresponding to each of the foveated area and the residual area, and having a size smaller than the size of the entire display area of ​​the display (1750) or a resolution smaller than the resolution of the display area.

[0265] The processor (1910) executing the renderer (1990) may obtain or generate a composite image to be displayed on the display (1750) by synthesizing an image corresponding to the foveated area and an image corresponding to the surrounding area. For example, the processor (1910) may perform upscaling to enlarge the image corresponding to the surrounding area to the size of the entire display area of ​​the display (1750). On the enlarged image, the processor (1910) may combine the image corresponding to the foveated area to generate a composite image to be displayed on the display (1750). Along the boundary line of the image corresponding to the foveated area, the processor (1910) may apply a visual effect, such as blur, to blend the enlarged image and the image corresponding to the foveated area.

[0266] Fig. 20 shows an example of a block diagram of an electronic device for displaying an image in virtual space.

[0267] Fig. 20 illustrates an example of a block diagram of an electronic device (e.g., the electronic device (1601) of Fig. 16) for displaying an image in a virtual space. In Fig. 20, an example of executing multiple programs / instructions for displaying an image in a virtual space is described. The multiple programs / instructions may all be executed in one processor (e.g., an AP) or may be executed by multiple processors (e.g., an AP, a GPU (graphics processing unit), an NPU (neural processing unit)). The meaning of being executable by the multiple processors means that some programs / instructions may be executed by a first processor and other some programs / instructions may be executed by a second processor different from the first processor.

[0268] Referring to FIG. 20, the electronic device (1601) may execute a virtual space manager (2050) (e.g., the virtual space manager (1951) of FIG. 19, CPM) to render an image in a virtual space. For the virtual space manager (2050), at least some of the descriptions of the virtual space manager (1951) of FIG. 19 may be referenced. The virtual space manager (2050) may include a platform for supporting a virtual space service. The virtual space manager (2050) may include a runtime service (2051) (e.g., open XR runtime), a panel renderer (2052) (e.g., 2D panel render), and an XR compositor (2053). The electronic device (1601) may execute at least one of a user's pose prediction function, a frame timing function, and / or a spatial input function based on the execution of the runtime service (2051). For the runtime service (2051), at least some of the descriptions of the runtime service (1952) of FIG. 19 may be referred to. The electronic device (1601) may display at least one image (video) on a panel (e.g., a 2D panel) to implement a virtual space through the display based on the execution of the panel rendering (2052). For example, the electronic device (1601) may display a rendering image corresponding to RGB information (2066) for the panel from the spatialization manager (2040) described below through the display (e.g., the display (1750)). The electronic device (1601) may synthesize an image of an actual area captured by a camera in the virtual space (hereinafter, a pass-through image) with a virtual area image based on the execution of the XR compositor (2053). For example, the electronic device (1601) can generate a composite image by merging the pass-through image and the virtual area image based on the execution of the XR synthesis unit (2053).The electronic device (1601) may transmit the generated composite image to a display buffer so that the composite image is displayed. The electronic device (1601) may identify a virtual space through a virtual space manager (2050) and display at least a portion of the virtual space on the display (1750). The virtual space manager (2050) may be referred to as a CPM. The electronic device (1601) may execute the virtual space manager (2050) to render an image corresponding to at least a portion of the virtual space.

[0269] According to one embodiment, the electronic device (1601) may execute a spatialization manager (2040). The spatialization manager (2040) may perform processes for displaying an image in a three-dimensional virtual space. The electronic device (1601) may perform preprocessing based on the execution of the spatialization manager (2040) so that the image can be rendered in the three-dimensional virtual space through the virtual space manager (2050). For example, the electronic device (1601) may perform at least some of the functions of the renderer (1990) of FIG. 19 based on the execution of the spatialization manager (2040). The electronic device (1601) may process image information provided by an application (e.g., an XR application (2010), an application (2020) that provides a general 2D screen other than XR, and an application that provides a system UI (2030)) based on the execution of the spatialization manager (2040). A spatialization manager (2040) (e.g., space flinger) may include a system scene manager (2041) (e.g., system scene), an input manager (2042) (e.g., input routing), and a lightweight rendering engine (2043) (e.g., impress engine). The system scene manager (2041) may be executed to display a system UI (2030). System UI-related information (2064) may be transmitted to the system scene manager (2041) from a program (e.g., API) that provides the system UI (2030). The system UI-related information (2064) may be obtained through a spatializer API and / or a same-process private API. The spatialization manager (2040) may determine the layout (e.g., location, display order) of the screen of the system UI (2030) in a three-dimensional space through pre-allocated resources.The system screen manager (2041) may transmit image information (2067) for rendering the screen of the system UI (2030) to the virtual space manager (2050) according to the layout. The input manager (2042) may be configured to process user input (e.g., user input on a system screen or an app screen). The impression engine (2043) may be a renderer for image generation (e.g., a lightweight renderer (1943)). For example, the impression engine (2043) may be used to display the system UI (2030). According to one embodiment, the spatialization manager (2040) may include a lightweight rendering engine (2043) for rendering the system UI. According to one embodiment, when the lightweight rendering engine (2043) does not have sufficient resources to render an avatar used in the HMD, at least one external rendering engine may be used. At this time, to resolve compatibility issues with external rendering (e.g., 3rd party engines), an external rendering engine support module may be added within the spatialization manager (2040).

[0270] According to one embodiment, the electronic device can execute an application. For example, in response to the execution of an XR application (2010) (e.g., an XR application (1942), a 3D game, an XR map, or other immersive application), the electronic device can execute a virtual space manager (2050). The electronic device (1601) can provide dual image information (2061) provided from the XR application (2010) to the virtual space manager (2050). In order to display an image in a three-dimensional space, the dual image information (2061) can include two pieces of image information that take binocular parallax into account. For example, the dual image information (2061) can include first image information for the user's left eye and second image information for the user's right eye for rendering in a three-dimensional virtual space. Hereinafter, in the present disclosure, the term dual image information is used to refer to image information for displaying images for both eyes in a three-dimensional space. In addition to the dual image information, the above dual image information may also include binocular image information, dual image information, dual image data, dual images, binocular image data, stereoscopic image information, 3D image information, spatial image information, spatial image data, 17D-3D conversion data, dimensional conversion image data, binocular parallax image data, and / or equivalent technical terms. The electronic device (1601) can generate a composite image by merging image layers through the virtual space manager (2050). The electronic device (1601) can transmit the generated composite image to a display buffer. The composite image can be displayed on the display (1750) of the electronic device (1601).

[0271] According to one embodiment, the electronic device can execute at least one application among an XR application (2010) and other applications (2020) (e.g., a first application (2020-1), a second application (2020-2), ..., an Nth application (2020-N)). According to one embodiment, the application (2020) can be configured to output image information for displaying a two-dimensional image. In other words, the application (2020) can provide a two-dimensional image. For example, the application (2020) can be a video application, a schedule application, or an application (2020) can be an Internet browser application. If it is assumed that in response to the execution of the application (2020), image information (2062) provided from the application (2020) is provided to the virtual space manager (2050). Since the image information (2062) only has x-coordinates and y-coordinates within a two-dimensional plane, it may be difficult to consider the chronological relationship (i.e., the distance from the user) between other applications centered on the user. Even when displaying an application (2020) that provides a general 2D screen, the electronic device (1601) may execute the spatialization manager (2040) to provide dual image information to the virtual space manager (2050). For example, based on the execution of the spatialization manager (2040), the electronic device (1601) may receive application-related information (2063) from the first application (2020-1). For example, the application-related information (2063) may include image information representing a two-dimensional image of the first application (2020-1) (e.g., information including RGB for each pixel) and / or content information in the first application (2020-1) (e.g., characteristics of content executed in the first application, type of content). Application related information (2063) can be obtained through the spatializer API.Based on the execution of the spatialization manager (2040), the electronic device (1601) can identify information about the location of the area to be rendered by the first application (2020-1) and the size of the area to be rendered (hereinafter, location information). Based on the execution of the spatialization manager (2040), the electronic device (1601) can generate dual image information (2065, e.g., RGBx2) that takes into account the user's binocular disparity through the image information and the location information. Based on the execution of the spatialization manager (2040), the electronic device (1601) can provide the dual image information (2065) to the virtual space manager (2050). By converting a simple two-dimensional image into the dual image information (2065), a problem that occurs when the image information (2062) is directly transmitted to the virtual space manager (2050) can be resolved. Additionally, since at least some of the functions for displaying images in a virtual space are performed by the spatialization manager (2040) instead of the virtual space manager (2050), the burden on the virtual space manager (2050) can be reduced.

[0272] The wearable device described above (e.g., the wearable device (100) of FIG. 3A) may include a memory (e.g., the memory (310) of FIG. 3A) that stores instructions and includes one or more storage media, one or more cameras (e.g., the one or more cameras (330) of FIG. 3A), a display assembly including a display (e.g., the one or more cameras (340) of FIG. 3A), a communication circuit (e.g., the communication circuit (320) of FIG. 3A), and at least one processor (e.g., the at least one processor (310) of FIG. 3A) that includes a processing circuit. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to acquire images of a space in front of the wearable device through the one or more cameras. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify whether a visual object having the shape of an electronic device (e.g., visual object (215) of FIG. 2) is included in at least some of the images. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device, based on identifying that the visual object is included in the at least some of the images, to request, via the communication circuit, from a server, information about external electronic devices having a type of electronic device corresponding to the visual object and registered in association with a user account of the wearable device. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to receive, via the communication circuit, information about one or more external electronic devices from the server.The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to determine an external electronic device corresponding to the visual object from among the one or more external electronic devices using the information received from the server. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display, through the display assembly, a user interface (UI) object (e.g., UI object (1305) of FIG. 13) associated with the visual object based on the determination. The UI object may be available to provide a service related to the external electronic device within the wearable device.

[0273] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to determine, using the information received from the server, among the one or more external electronic devices, an external electronic device located within a reference distance to the wearable device.

[0274] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to transmit, via the communication circuit, a signal to the one or more external electronic devices using the information received from the server, causing the external electronic devices to activate a sensor of the one or more external electronic devices. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to receive, via the communication circuit, sensing data acquired through the activated sensor of the one or more external electronic devices from the one or more external electronic devices. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to determine, using the sensing data, the external electronic device from among the one or more external electronic devices.

[0275] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display, through the display assembly, guidance to perform a user gesture associated with the external electronic device based on transmitting the signal to the one or more external electronic devices via the communication circuitry. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to receive, through the communication circuitry, the sensing data from the one or more external electronic devices. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to determine the external electronic device from among the one or more external electronic devices by identifying the user gesture using the sensing data.

[0276] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify distances between the wearable device and the one or more external electronic devices via the communication circuit using the information received from the server. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to determine the external electronic device from among the one or more external electronic devices based on a shortest distance between the wearable device and the external electronic device among the distances.

[0277] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to use the communication circuitry to identify strengths of wireless signals between the wearable device and the one or more external electronic devices using the information received from the server. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to determine, from among the one or more external electronic devices, the external electronic device corresponding to the visual object based on the strength of the wireless signal between the wearable device and the external electronic device that is the greatest among the strengths of the wireless signals.

[0278] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display, through the display assembly, guidance to turn on the external electronic device based on identifying that the visual object having the shape of a turned-off electronic device is included in at least some of the images. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to request, through the communication circuitry, from the server, information about the external electronic devices having a type of electronic device corresponding to the visual object and registered in association with the user account of the wearable device, based on identifying that the visual object having the shape of a turned-on electronic device is included in at least some of the images.

[0279] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to transmit, via the communication circuitry, a signal to the one or more external electronic devices, using the information received from the server, the signal causing the one or more external electronic devices to display a visual identifier through a display of the one or more external electronic devices. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to determine the external electronic device among the one or more external electronic devices based on identifying that the visual object including the visual identifier is included in at least some of the images.

[0280] For example, the visual identifier may include a barcode, an icon, a mark, and / or a lighting pattern of the display.

[0281] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify a first software application running within the external electronic device using the information received from the server. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify a second software application running within the external electronic device using at least a portion of the images. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to determine the external electronic device from among the one or more external electronic devices based on the second software application corresponding to the first software application.

[0282] For example, the wearable device may further include one or more other cameras arranged to face the eyes of a user of the wearable device when worn. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify that a gaze of the user of the wearable device, identified using images acquired through the one or more other cameras, is directed toward the visual object having the shape of an electronic device. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to request, through the communication circuitry, from a server, based on identifying the gaze of the user toward the visual object, information about external electronic devices that have a type of electronic device corresponding to the visual object and that are registered in association with a user account of the wearable device.

[0283] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to receive input via the UI object. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to provide a service related to the external electronic device by connecting to the external electronic device based on the input.

[0284] For example, the service related to the external electronic device may include a service for controlling the wearable device using the external electronic device, a service for displaying the screen of the external electronic device on the display assembly of the wearable device, and / or a service for providing an event detected within the external electronic device within the wearable device.

[0285] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display, through the display assembly, another UI object associated with the visual object based on determining another external electronic device from among the one or more external electronic devices using the information received from the server. The other UI object may be available to inquire whether to register the external electronic device as associated with the user account.

[0286] The method described above may be performed within a wearable device including one or more cameras, a display assembly including a display, and communication circuitry. The method may include an operation of acquiring images of a space in front of the wearable device through the one or more cameras. The method may include an operation of identifying whether a visual object having the shape of an electronic device is included in at least a portion of the images. The method may include an operation of requesting, through the communication circuitry, from a server, based on identifying that the visual object is included in the at least a portion of the images, information about external electronic devices having a type of electronic device corresponding to the visual object and registered in association with a user account of the wearable device. The method may include an operation of receiving, through the communication circuitry, information about one or more external electronic devices from the server. The method may include an operation of determining, from among the one or more external electronic devices, an external electronic device corresponding to the visual object using the information received from the server. The method may include an operation of displaying a user interface (UI) object associated with the visual object through the display assembly based on the determination. The UI object may be available to provide a service related to the external electronic device within the wearable device.

[0287] For example, the method may include an operation of determining, from among the one or more external electronic devices, an external electronic device located within a reference distance from the wearable device, using the information received from the server.

[0288] For example, the method may include an operation of using the information received from the server to transmit a signal to the one or more external electronic devices via the communication circuit, the signal causing the one or more external electronic devices to activate a sensor of the one or more external electronic devices. The method may include an operation of receiving, from the one or more external electronic devices via the communication circuit, sensing data acquired via the activated sensor of the one or more external electronic devices. The method may include an operation of using the sensing data to determine the external electronic device from among the one or more external electronic devices.

[0289] For example, the method may include an operation of displaying, through the display assembly, guidance to perform a user gesture related to the external electronic device based on transmitting the signal to the one or more external electronic devices through the communication circuit. The method may include an operation of receiving, through the communication circuit, the sensing data from the one or more external electronic devices. The method may include an operation of determining the external electronic device from among the one or more external electronic devices by identifying the user gesture using the sensing data.

[0290] For example, the method may include an operation of identifying distances between the wearable device and the one or more external electronic devices through the communication circuit using the information received from the server. The method may include an operation of determining the external electronic device from among the one or more external electronic devices based on a shortest distance between the wearable device and the external electronic device among the distances.

[0291] For example, the method may include an operation of identifying, using the information received from the server, the strength of wireless signals between the wearable device and the one or more external electronic devices using the communication circuit. The method may include an operation of determining, from among the one or more external electronic devices, the external electronic device corresponding to the visual object based on the strength of the wireless signal between the wearable device and the external electronic device that is the greatest among the strengths of the wireless signals.

[0292] For example, the method may include an action of displaying, through the display assembly, guidance to turn on the external electronic device based on identifying that the visual object having the shape of a turned-off electronic device is included in at least a portion of the images. The method may include an action of requesting, through the communication circuit, to the server, information about the external electronic devices having a type of electronic device corresponding to the visual object and registered in association with the user account of the wearable device, based on identifying that the visual object having the shape of a turned-on electronic device is included in at least a portion of the images.

[0293] For example, the method may include an action of using the information received from the server to transmit a signal to the one or more external electronic devices via the communication circuit, the signal causing the one or more external electronic devices to display a visual identifier through a display of the one or more external electronic devices. The method may include an action of determining the external electronic device from among the one or more external electronic devices based on identifying that the visual object including the visual identifier is included in at least a portion of the images.

[0294] For example, the visual identifier may include a barcode, an icon, a mark, and / or a lighting pattern of the display.

[0295] For example, the method may include an operation of identifying a first software application running within the external electronic device using the information received from the server. The method may include an operation of identifying a second software application running within the external electronic device using at least a portion of the images. The method may include an operation of determining the external electronic device from among the one or more external electronic devices based on the second software application corresponding to the first software application.

[0296] For example, the wearable device may further include one or more other cameras arranged to face the eyes of a user of the wearable device when worn. The method may include an operation of identifying that a gaze of the user of the wearable device, identified using images acquired through the one or more other cameras, is directed toward the visual object having the shape of an electronic device. The method may include an operation of requesting, through the communication circuit, information about external electronic devices that have a type of electronic device corresponding to the visual object and are registered in association with a user account of the wearable device based on the identification of the gaze of the user toward the visual object.

[0297] For example, the method may include an operation of receiving an input through the UI object. The method may include an operation of providing a service related to the external electronic device by connecting to the external electronic device based on the input.

[0298] For example, the service related to the external electronic device may include a service for controlling the wearable device using the external electronic device, a service for displaying the screen of the external electronic device on the display assembly of the wearable device, and / or a service for providing an event detected within the external electronic device within the wearable device.

[0299] For example, the method may include an operation of displaying, through the display assembly, another UI object associated with the visual object based on determining another external electronic device from among the one or more external electronic devices using the information received from the server. The other UI object may be available to inquire whether to register the external electronic device as associated with the user account.

[0300] The non-transitory computer-readable storage medium described above may store one or more programs. The one or more programs may include instructions that, when executed by a wearable device including one or more cameras, a display assembly including a display, and communication circuitry, cause the wearable device to acquire images of a space in front of the wearable device through the one or more cameras. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to identify whether a visual object having the shape of an electronic device is included in at least a portion of the images. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to request, through the communication circuitry, from a server information about external electronic devices having a type of electronic device corresponding to the visual object and registered in association with a user account of the wearable device, based on identifying that the visual object is included in the at least a portion of the images. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to receive, through the communication circuitry, information about one or more external electronic devices from the server. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to determine, from among the one or more external electronic devices, an external electronic device corresponding to the visual object using the information received from the server.The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to display, through the display assembly, a user interface (UI) object associated with the visual object based on the determination. The UI object may be available to provide a service related to the external electronic device within the wearable device.

[0301] For example, the one or more programs may include instructions that cause the wearable device, when executed by the wearable device, to determine, from among the one or more external electronic devices, an external electronic device located within a reference distance to the wearable device, using the information received from the server.

[0302] For example, the one or more programs may include instructions that cause the wearable device, when executed by the wearable device, to transmit a signal to the one or more external electronic devices via the communication circuit using the information received from the server, causing the external electronic devices to activate a sensor of the one or more external electronic devices. The one or more programs may include instructions that cause the wearable device, when executed by the wearable device, to receive, via the communication circuit, sensing data acquired via the activated sensor of the one or more external electronic devices from the one or more external electronic devices. The one or more programs may include instructions that cause the wearable device, when executed by the wearable device, to determine the external electronic device from among the one or more external electronic devices using the sensing data.

[0303] For example, the one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to display, through the display assembly, guidance to perform a user gesture associated with the external electronic device based on transmitting the signal to the one or more external electronic devices via the communication circuitry. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to receive, through the communication circuitry, the sensing data from the one or more external electronic devices. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to determine the external electronic device from among the one or more external electronic devices by identifying the user gesture using the sensing data.

[0304] For example, the one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to identify distances between the wearable device and the one or more external electronic devices through the communication circuit using the information received from the server. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to determine the external electronic device from among the one or more external electronic devices based on a shortest distance between the wearable device and the external electronic device among the distances.

[0305] For example, the one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to identify, using the communication circuitry, the strength of wireless signals between the wearable device and the one or more external electronic devices using the information received from the server. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to determine, from among the one or more external electronic devices, the external electronic device corresponding to the visual object based on the strength of the wireless signal between the wearable device and the external electronic device that is the greatest among the strengths of the wireless signals.

[0306] For example, the one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to display, through the display assembly, guidance to turn on the external electronic device based on identifying that the visual object having the shape of a turned-off electronic device is included in at least some of the images. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to request, through the communication circuitry, information about the external electronic devices that have a type of electronic device corresponding to the visual object and that are registered in association with the user account of the wearable device, based on identifying that the visual object having the shape of a turned-on electronic device is included in at least some of the images.

[0307] For example, the one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to transmit, via the communication circuitry, a signal to the one or more external electronic devices, using the information received from the server, causing the one or more external electronic devices to display a visual identifier through a display of the one or more external electronic devices. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to determine the external electronic device from among the one or more external electronic devices based on identifying that the visual object including the visual identifier is included in at least a portion of the images.

[0308] For example, the visual identifier may include a barcode, an icon, a mark, and / or a lighting pattern of the display.

[0309] For example, the one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to identify a first software application running within the external electronic device using the information received from the server. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to identify a second software application running within the external electronic device using at least a portion of the images. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to determine the external electronic device from among the one or more external electronic devices based on the second software application corresponding to the first software application.

[0310] For example, the wearable device may further include one or more other cameras arranged to face the eyes of a user of the wearable device when worn. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to identify that the gaze of the user of the wearable device, identified using images acquired through the one or more other cameras, is directed toward the visual object having the shape of an electronic device. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to request, through the communication circuitry, information about external electronic devices that have a type of electronic device corresponding to the visual object and that are registered in association with a user account of the wearable device based on the identification of the gaze of the user toward the visual object.

[0311] For example, the one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to receive input through the UI object. The one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to provide a service related to the external electronic device by connecting to the external electronic device based on the input.

[0312] For example, the service related to the external electronic device may include a service for controlling the wearable device using the external electronic device, a service for displaying the screen of the external electronic device on the display assembly of the wearable device, and / or a service for providing an event detected within the external electronic device within the wearable device.

[0313] For example, the one or more programs may include instructions that, when executed by the wearable device, cause the wearable device to display, through the display assembly, another UI object associated with the visual object based on determining another external electronic device from among the one or more external electronic devices using the information received from the server.

Claims

1. In a wearable device (100), A memory (310) storing instructions and including one or more storage media; One or more cameras (330); A display assembly (340) including a display; Communication circuit (320); and At least one processor (300) comprising a processing circuit, The above instructions, when individually or collectively executed by the at least one processor (300), Through one or more of the above cameras (330), images of the space in front of the wearable device (100) are acquired, Identifying whether a visual object (215) having the shape of an electronic device is included within at least some of the images; Based on identifying that the visual object (215) is included in at least a portion of the images, requesting information about external electronic devices registered in association with a user account of the wearable device (100) and having a type of electronic device corresponding to the visual object (215) to the server (120) through the communication circuit (320), Through the above communication circuit (320), information about one or more external electronic devices is received from the server (120), Using the information received from the server (120), an external electronic device (301) corresponding to the visual object (215) is determined among one or more external electronic devices, and Based on the above decision, the UI (user interface) object (1305) linked to the visual object (215) is displayed through the display assembly (340). causing the above wearable device (100), The above UI object (1305) is Available to provide services related to the external electronic device (301) within the wearable device (100). Wearable device (100).

2. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (300), Using the information received from the server (120), determine an external electronic device (301) located within a reference distance from the wearable device (100) among the one or more external electronic devices. causing the above wearable device (100), Wearable device (100).

3. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (300), Using the information received from the server (120), a signal is transmitted to the one or more external electronic devices through the communication circuit (320) to cause the external electronic devices to activate a sensor of the one or more external electronic devices, Through the above communication circuit (320), sensing data obtained through the activated sensors of the one or more external electronic devices is received from the one or more external electronic devices, and Using the sensing data, determine the external electronic device (301) among the one or more external electronic devices. causing the above wearable device (100), Wearable device (100).

4. In claim 3, The above instructions, when individually or collectively executed by the at least one processor (300), Based on transmitting the above signal, a guidance (1105, 1130) is displayed to perform a user gesture related to the external electronic device (301) through the display assembly (340). Through the above communication circuit (320), the sensing data is received from one or more external electronic devices, and By using the sensing data, by identifying the user gesture, the external electronic device (301) is determined among the one or more external electronic devices. causing the above wearable device (100), Wearable device (100).

5. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (300), Using the information received from the server (120), the distances between the wearable device (100) and the one or more external electronic devices (301) are identified through the communication circuit (320), and Based on the shortest distance between the wearable device (100) and the external electronic device (301) among the above distances, the external electronic device (301) is determined among the one or more external electronic devices. causing the above wearable device (100), Wearable device (100).

6. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (300), Using the information received from the server (120), the strength of wireless signals between the wearable device (100) and the one or more external electronic devices (301) is identified through the communication circuit (320), and Based on the strength of the wireless signal between the wearable device (100) and the external electronic device (301) that is the greatest among the strengths of the wireless signals, the external electronic device (301) is determined among the one or more external electronic devices. causing the above wearable device (100), Wearable device (100).

7. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (300), Based on identifying that the visual object (215) having the shape of a turned-off electronic device is included in at least a portion of the images, a guidance (805) is displayed to turn on the external electronic device (301) through the display assembly (340), and Based on identifying that the visual object (215) having the shape of a turned-on electronic device is included in at least a portion of the images, requesting the server (120) through the communication circuit (320) the information about external electronic devices having a type of electronic device corresponding to the visual object (215) and registered in association with the user account of the wearable device (100). causing the above wearable device (100), Wearable device (100).

8. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (300), Using the information received from the server (120), a signal is transmitted to the one or more external electronic devices through the communication circuit (320) to cause the one or more external electronic devices to display a visual identifier (1205) through a display of the one or more external electronic devices, and Determine the external electronic device (301) among the one or more external electronic devices based on identifying that the visual object (215) including the visual identifier (1205) is included in at least some of the images; causing the above wearable device (100), Wearable device (100).

9. In claim 8, The above visual identifier (1205) is, including a barcode, icon, mark, and / or lighting pattern of the display; Wearable device (100).

10. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (300), Using the information received from the server (120), the first software application running in the external electronic device (301) is identified, Using at least some of the above images, identifying a second software application running within the external electronic device (301), and Based on the second software application corresponding to the first software application, determine the external electronic device (301) among the one or more external electronic devices. causing the above wearable device (100), Wearable device (100).

11. In claim 1, Further comprising one or more other cameras (330) arranged to face the eyes of the user of the wearable device (100) when worn; The above instructions, when individually or collectively executed by the at least one processor (300), Identifying that the user's gaze is directed toward the visual object (215) having the shape of an electronic device using images acquired through one or more of the other cameras (330), and Based on identifying the user's gaze toward the visual object (215), requesting information about external electronic devices registered in connection with the user account of the wearable device (100) and having a type of electronic device corresponding to the visual object (215) to the server (120) through the communication circuit (320). causing the above wearable device (100), Wearable device (100).

12. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (300), Receiving input (1310) through the above UI object (1305), and Based on the above input, by connecting to the external electronic device (301), to provide the service related to the external electronic device (301). causing the above wearable device (100), Wearable device (100).

13. In claim 1, The above service related to the above external electronic device (301) is, A service for controlling the wearable device (100) using the external electronic device (301), a service for displaying the screen of the external electronic device (301) through the display assembly (340) of the wearable device (100), and / or a service for providing an event detected within the external electronic device (301) within the wearable device (100). Wearable device (100).

14. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (300), Using the information received from the server (120), based on determining another external electronic device among the one or more external electronic devices, display another UI object (1505) associated with the visual object (215) through the display assembly (340). causing the above wearable device (100), The other UI object (1505) above is, Available to inquire whether to register the above external electronic device (301) in conjunction with the above user account. Wearable device (100).

15. A method executed within a wearable device (100) comprising one or more cameras (330), a display assembly (340) including a display, and a communication circuit (320), An operation of acquiring images of the space in front of the wearable device (100) through one or more of the cameras (330); An operation of identifying whether a visual object (215) having the shape of an electronic device is included in at least some of the images; An operation of requesting information about external electronic devices registered in connection with a user account of the wearable device (100) having a type of electronic device corresponding to the visual object (215) to a server (120) through the communication circuit (320) based on identifying that the visual object (215) is included in at least a portion of the images, An operation of receiving information about one or more external electronic devices from the server (120) through the communication circuit (320); An operation of determining an external electronic device (301) corresponding to the visual object (215) among one or more external electronic devices using the information received from the server (120), and Based on the above decision, an operation of displaying a UI (user interface) object (1305) linked to the visual object (215) through the display assembly (340) is included. The above UI object (1305) is Available to provide services related to the external electronic device (301) within the wearable device (100). method.

Images