Electronic device and method for displaying visual object on basis of object recognition

Abstract

This wearable device may comprise: a memory for storing instructions; a display; a communication circuit; and at least one processor. The instructions, when executed individually or collectively by the at least one processor, may cause the wearable device to: connect to one or more external electronic devices through the communication circuit; on the basis of identifying a designated event, identify a target object among one or more external objects within a field of view (FoV) of the wearable device; on the basis of the size of the target object, identify an external electronic device among the one or more external electronic devices; receive, from the external electronic device, information for displaying a screen associated with the external electronic device; and display, through the display, a visual object generated on the basis of the information and at least partially overlapping the target object.

Description

Electronic device and method for displaying visual objects based on object recognition

[0001] The descriptions below relate to electronic devices and methods for displaying visual objects based on object recognition.

[0002] To provide an enhanced user experience, electronic devices are being developed that provide extended reality services that display computer-generated information in conjunction with external objects in the real world or virtual objects in the virtual world. The electronic devices may include wearable devices that can be worn by a user. For example, the electronic devices may include user equipment, AR glasses, VR glasses, and / or a head-mounted device (HMD) (e.g., a video see-through (VST) HMD, an optical see-through (OST) HMD).

[0003] A wearable device may include a memory storing instructions and including one or more storage media. The wearable device may include a display. The wearable device may include a communication circuit. The wearable device may include at least one processor including a processing circuit. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to connect with one or more external electronic devices via the communication circuit. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify a target object among one or more external objects within a field of view (FoV) of the wearable device based on identifying a designated event. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify an external electronic device among the one or more external electronic devices based on an attribute of the target object. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to receive, from the external electronic device, information for displaying a screen associated with the external electronic device. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display, through the display, a visual object generated based on the information and at least partially superimposed on the target object.A method performed by a wearable device may include an operation of connecting with one or more external electronic devices. The method may include an operation of identifying a target object among one or more external objects within a field of view (FoV) of the wearable device based on identifying a specified event. The method may include an operation of identifying an external electronic device among the one or more external electronic devices based on a size of the target object. The method may include an operation of receiving, from the external electronic device, information for displaying a screen associated with the external electronic device. The method may include an operation of displaying a visual object generated based on the information and at least partially superimposed on the target object.

[0004] A non-transitory computer-readable storage medium may store one or more programs comprising instructions that, when individually or collectively executed by at least one processor of a wearable device including a display and a communication circuit, cause the wearable device to connect with one or more external electronic devices via the communication circuit. The non-transitory computer-readable storage medium may store one or more programs comprising instructions that, when individually or collectively executed by the at least one processor, cause the wearable device to identify a target object among one or more external objects within a field of view (FoV) of the wearable device based on identifying a designated event. The non-transitory computer-readable storage medium may store one or more programs including instructions that, when individually or collectively executed by the at least one processor, cause the wearable device to identify an external electronic device among the one or more external electronic devices based on an attribute of the target object. The non-transitory computer-readable storage medium may store one or more programs including instructions that, when individually or collectively executed by the at least one processor, cause the wearable device to receive, from the external electronic device, information for displaying a screen associated with the external electronic device.The non-transitory computer-readable storage medium may store one or more programs including instructions that, when individually or collectively executed by the at least one processor, cause the wearable device to display, through the display, a visual object generated based on the information and at least partially superimposed on the target object.

[0005] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments.

[0006] FIG. 2A illustrates an example of a perspective view of a wearable device according to various embodiments.

[0007] FIG. 2b illustrates an example of one or more hardware elements arranged within a wearable device according to various embodiments.

[0008] FIGS. 3A and 3B illustrate examples of the appearance of a wearable device according to various embodiments.

[0009] FIG. 4 illustrates an example of a method for a wearable device to display information obtained from an external electronic device with respect to an external object.

[0010] FIG. 5 illustrates an example of an external electronic device connected to a wearable device and an external object identified by the wearable device.

[0011] Figure 6 illustrates an example of an operational flow for a method in which a wearable device displays information obtained from an external electronic device on an external object.

[0012] FIGS. 7A through 7C illustrate examples of a method for identifying an external electronic device corresponding to a target object identified based on a specified event.

[0013] FIG. 8 illustrates examples of a method for displaying a visual object generated based on information obtained from an external electronic device on an area of ​​an external object.

[0014] Figure 9 illustrates an example of a method for displaying visual objects on multiple subregions of an external object.

[0015] Figure 10a illustrates an example of how to change the visual objects displayed based on user input.

[0016] Figure 10b illustrates an example of how to change the displayed visual object based on a change in the size of an external object.

[0017] FIG. 11 illustrates an example of an operational flow for a method in which a wearable device displays a visual object generated based on information acquired from an external electronic device on an external object.

[0018] The terms used in this disclosure are used only to describe specific embodiments and may not be intended to limit the scope of other embodiments. The singular expression may include plural expressions unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as commonly understood by those of ordinary skill in the art described in this disclosure. Terms defined in general dictionaries among the terms used in this disclosure may be interpreted as having the same or similar meaning in the context of the relevant technology, and shall not be interpreted in an idealized or overly formal sense unless explicitly defined in this disclosure. In some cases, even if a term is defined in this disclosure, it cannot be interpreted to exclude embodiments of the present disclosure.

[0019] The various embodiments of the present disclosure described below illustrate a hardware-based approach as an example. However, since the various embodiments of the present disclosure include techniques utilizing both hardware and software, the various embodiments of the present disclosure do not exclude a software-based approach.

[0020] In addition, in the present disclosure, expressions such as "more than" or "less than" may be used to determine whether a specific condition is satisfied or fulfilled. However, this is merely a description for expressing an example and does not exclude descriptions such as "more than" or "less than." Conditions described as "more than" may be replaced with "more than," conditions described as "less than," and conditions described as "more than and less than" may be replaced with "more than and less than." In addition, hereinafter, "A" to "B" mean at least one of the elements from A (including A) to B (including B).

[0021] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments.

[0022] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0023] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or calculations. According to one embodiment, as at least a part of the data processing or calculations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or a secondary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor)) that can operate independently or together therewith. For example, if the electronic device (101) includes a main processor (121) and a secondary processor (123), the secondary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a specified function. The secondary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0024] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0025] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0026] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0027] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0028] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0029] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. In one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0030] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0031] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0032] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0033] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0034] A haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. In one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0035] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0036] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least a part of a power management integrated circuit (PMIC).

[0037] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0038] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0039] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0040] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas by, for example, the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0041] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0042] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0043] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0044] FIG. 2A illustrates an example of a perspective view of a wearable device according to various embodiments. FIG. 2B illustrates an example of one or more hardware components arranged within a wearable device according to various embodiments.

[0045] According to one embodiment, the wearable device (103) may have the form of glasses that are wearable on a body part of the user (e.g., the head). The wearable device (103) of FIGS. 2A and 2B may be an example of the electronic device (101) of FIG. 1. The wearable device (103) may include a head-mounted display (HMD). For example, the housing of the wearable device (103) may include a flexible material, such as rubber and / or silicone, that is configured to fit closely to a portion of the user's head (e.g., a portion of the face surrounding both eyes). For example, the housing of the wearable device (103) may include one or more straps that are capable of being twined around the user's head, and / or one or more temples that are detachably attachable to the ears of the head.

[0046] Referring to FIG. 2A, according to one embodiment, a wearable device (103) may include at least one display (250) and a frame (200) supporting at least one display (250).

[0047] According to one embodiment, a wearable device (103) can be worn on a part of a user's body. The wearable device (103) can provide extended reality (XR) to a user wearing the wearable device (103). For example, the extended reality can provide augmented reality (AR), virtual reality (VR), or mixed reality (MR) that combines augmented reality and virtual reality. For example, the wearable device (103) can display a virtual reality image provided from at least one optical device (282, 284) of FIG. 2B on at least one display (250) in response to a user's designated gesture acquired through the motion recognition cameras (260-2, 263) of FIG. 2B.

[0048] According to one embodiment, at least one display (250) may provide visual information to a user. For example, at least one display (250) may include a transparent or translucent lens. At least one display (250) may include a first display (250-1) and / or a second display (250-2) spaced apart from the first display (250-1). For example, the first display (250-1) and the second display (250-2) may be positioned at positions corresponding to the user's left and right eyes, respectively.

[0049] Referring to FIG. 2B, at least one display (250) can provide visual information transmitted from external light to a user through a lens included in at least one display (250), and other visual information distinct from the visual information. The lens can be formed based on at least one of a Fresnel lens, a pancake lens, or a multi-channel lens. For example, at least one display (250) can include a first surface (231) and a second surface (232) opposite to the first surface (231). A display area can be formed on the second surface (232) of at least one display (250). When a user wears the wearable device (103), external light can be transmitted to the user by being incident on the first surface (231) and transmitted through the second surface (232). As another example, at least one display (250) can display an augmented reality image combined with a virtual reality image provided from at least one optical device (282, 284) on a real screen transmitted through external light, in a display area formed on the second surface (232).

[0050] In one embodiment, at least one display (250) may include at least one waveguide (233, 234) that diffracts light emitted from at least one optical device (282, 284) and transmits the diffracted light to a user. The at least one waveguide (233, 234) may be formed based on at least one of glass, plastic, or polymer. A nano-pattern may be formed on at least a portion of the exterior or interior of the at least one waveguide (233, 234). The nano-pattern may be formed based on a grating structure having a polygonal and / or curved shape. Light incident on one end of the at least one waveguide (233, 234) may be propagated to the other end of the at least one waveguide (233, 234) by the nano-pattern. At least one waveguide (233, 234) may include at least one diffractive element (e.g., a diffractive optical element (DOE), a holographic optical element (HOE)) and at least one reflective element (e.g., a reflective mirror). For example, at least one waveguide (233, 234) may be arranged within the wearable device (103) to guide a screen displayed by at least one display (250) to the user's eyes. For example, the screen may be transmitted to the user's eyes based on total internal reflection (TIR) ​​occurring within the at least one waveguide (233, 234).

[0051] The wearable device (103) can analyze an object included in a real image collected through a shooting camera (260-4), combine a virtual object corresponding to an object to be provided with augmented reality among the analyzed objects, and display the virtual object on at least one display (250). The virtual object can include at least one of text and an image regarding various information related to the object included in the real image. The wearable device (103) can analyze the object based on a multi-camera such as a stereo camera. For the object analysis, the wearable device (103) can perform spatial recognition (e.g., simultaneous localization and mapping (SLAM)) using a multi-camera and / or time-of-flight (ToF). A user wearing the wearable device (103) can view an image displayed on at least one display (250).

[0052] According to one embodiment, the frame (200) may be configured as a physical structure that allows the wearable device (103) to be worn on the user's body. According to one embodiment, the frame (200) may be configured so that, when the user wears the wearable device (103), the first display (250-1) and the second display (250-2) can be positioned corresponding to the user's left and right eyes. The frame (200) may support at least one display (250). For example, the frame (200) may support the first display (250-1) and the second display (250-2) to be positioned corresponding to the user's left and right eyes.

[0053] Referring to FIG. 2A, the frame (200) may include a region (220) that is in contact with at least a portion of a user's body when the user wears the wearable device (103). For example, the region (220) of the frame (200) that is in contact with a portion of the user's body may include a region that is in contact with a portion of the user's nose, a portion of the user's ear, and a portion of the side of the user's face that the wearable device (103) makes contact with. According to one embodiment, the frame (200) may include a nose pad (210) that is in contact with a portion of the user's body. When the wearable device (103) is worn by the user, the nose pad (210) may be in contact with a portion of the user's nose. The frame (200) may include a first temple (204) and a second temple (205) that are in contact with another portion of the user's body that is distinct from the portion of the user's body.

[0054] For example, the frame (200) may include a first rim (201) that surrounds at least a portion of the first display (250-1), a second rim (202) that surrounds at least a portion of the second display (250-2), a bridge (203) that is disposed between the first rim (201) and the second rim (202), a first pad (211) that is disposed along a portion of the edge of the first rim (201) from one end of the bridge (203), a second pad (212) that is disposed along a portion of the edge of the second rim (202) from the other end of the bridge (203), a first temple (204) that extends from the first rim (201) and is fixed to a portion of the wearer's ear, and a second temple (205) that extends from the second rim (202) and is fixed to a portion of the ear opposite the ear. The first pad (211) and the second pad (212) may be in contact with a portion of the user's nose, and the first temple (204) and the second temple (205) may be in contact with a portion of the user's face and a portion of the user's ear. The temples (204, 205) may be rotatably connected to the rim through the hinge units (206, 207) of FIG. 2B. The first temple (204) may be rotatably connected to the first rim (201) through the first hinge unit (206) disposed between the first rim (201) and the first temple (204). The second temple (205) may be rotatably connected to the second rim (202) through the second hinge unit (207) disposed between the second rim (202) and the second temple (205). According to one embodiment, the wearable device (103) can identify an external object (e.g., a user's fingertip) touching the frame (200) and / or a gesture performed by the external object by using a touch sensor, a grip sensor, and / or a proximity sensor formed on at least a portion of a surface of the frame (200).

[0055] According to one embodiment, the wearable device (103) may include hardwares (e.g., hardwares to be described later in FIG. 5) that perform various functions. For example, the hardwares may include a battery module (270), an antenna module (275), at least one optical device (282, 284), speakers (e.g., speakers 255-1, 255-2), a microphone (e.g., microphones 265-1, 265-2, 265-3), a light-emitting module (not shown), and / or a printed circuit board (PCB) (290) (e.g., a printed circuit board). The various hardwares may be arranged within the frame (200).

[0056] According to one embodiment, the microphones (e.g., microphones 265-1, 265-2, 265-3) of the wearable device (103) may be disposed on at least a portion of the frame (200) to acquire sound signals. A first microphone (265-1) disposed on the bridge (203), a second microphone (265-2) disposed on the second rim (202), and a third microphone (265-3) disposed on the first rim (201) are illustrated in FIG. 2B , but the number and arrangement of the microphones (265) are not limited to the embodiment of FIG. 2B . When the number of microphones (265) included in the wearable device (103) is two or more, the wearable device (103) may identify the direction of the sound signal by using a plurality of microphones disposed on different portions of the frame (200).

[0057] According to one embodiment, at least one optical device (282, 284) can project a virtual object onto at least one display (250) to provide various image information to a user. For example, at least one optical device (282, 284) can be a projector. At least one optical device (282, 284) can be disposed adjacent to at least one display (250) or can be included within at least one display (250) as a part of at least one display (250). According to one embodiment, the wearable device (103) can include a first optical device (282) corresponding to a first display (250-1) and a second optical device (284) corresponding to a second display (250-2). For example, at least one optical device (282, 284) may include a first optical device (282) disposed at an edge of a first display (250-1) and a second optical device (284) disposed at an edge of a second display (250-2). The first optical device (282) may transmit light to a first waveguide (233) disposed on the first display (250-1), and the second optical device (284) may transmit light to a second waveguide (234) disposed on the second display (250-2).

[0058] In one embodiment, the camera (260) may include a recording camera (260-4), an eye tracking camera (ET CAM) (260-1), and / or a motion recognition camera (260-2, 260-3). The recording camera (260-4), the eye tracking camera (260-1), and the motion recognition cameras (260-2, 260-3) may be positioned at different locations on the frame (200) and may perform different functions. The eye tracking camera (260-1) may output data indicating the position or gaze of the eyes of a user wearing the wearable device (103). For example, the wearable device (103) may detect the gaze from an image including the user's pupils obtained through the eye tracking camera (260-1). The wearable device (103) can perform gaze interaction with at least one object using the user's gaze acquired through the gaze tracking camera (260-1). The wearable device (103) can express a portion corresponding to the eye of an avatar representing the user in a virtual space using the user's gaze acquired through the gaze tracking camera (260-1). The wearable device (103) can render an image (or screen) displayed on at least one display (250) based on the position of the user's eyes. For example, the visual quality (e.g., resolution, brightness, saturation, grayscale, PPI) of a first region related to the gaze and a second region distinguished from the first region within the image may be different from each other. For example, if the wearable device (103) supports an iris recognition function, user authentication can be performed based on iris information acquired using the gaze tracking camera (260-1).An example in which the gaze tracking camera (260-1) is positioned toward the user's right eye is illustrated in FIG. 2B, but the embodiment is not limited thereto, and the gaze tracking camera (260-1) may be positioned solely toward the user's left eye, or may be positioned toward both eyes.

[0059] In one embodiment, the capturing camera (260-4) can capture an actual image or background to be aligned with a virtual image in order to implement augmented reality or mixed reality content. The capturing camera (260-4) can capture an image of a specific object existing at a location viewed by the user and provide the image to at least one display (250). The at least one display (250) can display a single image in which information about an actual image or background including an image of the specific object acquired using the capturing camera (260-4) is superimposed on a virtual image provided through at least one optical device (282, 284). The wearable device (103) can compensate for depth information (e.g., the distance between the wearable device (103) and an external object acquired using a depth sensor) using the image acquired using the capturing camera (260-4). The wearable device (103) can perform object recognition using the image acquired using the capturing camera (260-4). The wearable device (103) may perform a pass-through function to display an image acquired through a photographing camera (260-4) by overlaying it on at least a portion of a screen representing a virtual space on at least one display (250). In one embodiment, the photographing camera (260-4) may be positioned on a bridge (203) positioned between a first rim (201) and a second rim (202).

[0060] The gaze tracking camera (260-1) can implement more realistic augmented reality by tracking the gaze of a user wearing the wearable device (103) and thereby matching the user's gaze with visual information provided to at least one display (250). For example, when the wearable device (103) looks straight ahead, the wearable device (103) can naturally display environmental information related to the user's front at a location where the user is located on at least one display (250). The gaze tracking camera (260-1) can be configured to capture an image of the user's pupil to determine the user's gaze. For example, the gaze tracking camera (260-1) can receive gaze detection light reflected from the user's pupil and track the user's gaze based on the position and movement of the received gaze detection light. In one embodiment, the gaze tracking camera (260-1) can be positioned at positions corresponding to the user's left and right eyes. For example, the gaze tracking camera (260-1) may be positioned within the first rim (201) and / or the second rim (202) to face the direction in which the user wearing the wearable device (103) is positioned.

[0061] The motion recognition camera (260-2, 260-3) can recognize the movement of the user's entire body, such as the user's torso, hand, or face, or a part of the body, and thereby provide a specific event on a screen provided on at least one display (250). The motion recognition camera (260-2, 260-3) can recognize the user's motion (gesture recognition), obtain a signal corresponding to the motion, and provide a display corresponding to the signal on at least one display (250). The wearable device (103) can identify the signal corresponding to the motion, and perform a designated function based on the identification. The motion recognition camera (260-2, 260-3) can be used to perform a spatial recognition function using SLAM and / or a depth map for 6 degrees of freedom pose (6 dof pose). The wearable device (103) can perform gesture recognition and / or object tracking functions using motion recognition cameras (260-2, 260-3). In one embodiment, the motion recognition cameras (260-2, 260-3) can be positioned on the first rim (201) and / or the second rim (202).

[0062] The camera (260) included in the wearable device (103) is not limited to the above-described gaze tracking camera (260-1) and motion recognition cameras (260-2, 260-3). For example, the wearable device (103) can identify an external object included in the FoV using a camera positioned toward the user's FoV. The wearable device (103) can identify an external object based on a sensor for identifying the distance between the wearable device (103) and the external object, such as a depth sensor and / or a time of flight (ToF) sensor. The camera (260) positioned toward the FoV can support an autofocus function and / or an optical image stabilization (OIS) function. For example, the wearable device (103) may include a camera (260) (e.g., a face tracking (FT) camera) positioned toward the face to obtain an image including the face of a user wearing the wearable device (103).

[0063] Although not shown, in one embodiment, the wearable device (103) may further include a light source (e.g., an LED) that emits light toward a subject (e.g., a user's eyes, face, and / or an external object within the FoV) being photographed using the camera (260). The light source may include an infrared wavelength LED. The light source may be disposed on at least one of the frame (200) and the hinge units (206, 207).

[0064] According to one embodiment, the battery module (270) may supply power to electronic components of the wearable device (103). In one embodiment, the battery module (270) may be disposed within the first temple (204) and / or the second temple (205). For example, the battery module (270) may be a plurality of battery modules (270). The plurality of battery modules (270) may be disposed within each of the first temple (204) and the second temple (205). In one embodiment, the battery module (270) may be disposed at an end of the first temple (204) and / or the second temple (205).

[0065] The antenna module (275) can transmit signals or power to the outside of the wearable device (103), or receive signals or power from the outside. In one embodiment, the antenna module (275) can be positioned within the first temple (204) and / or the second temple (205). For example, the antenna module (275) can be positioned close to one surface of the first temple (204) and / or the second temple (205).

[0066] The speaker (255) can output an audio signal to the outside of the wearable device (103). The audio output module may be referred to as a speaker. In one embodiment, the speaker (255) may be positioned within the first temple (204) and / or the second temple (205) so as to be positioned adjacent to the ear of a user wearing the wearable device (103). For example, the speaker (255) may include a second speaker (255-2) positioned within the first temple (204) and thus positioned adjacent to the user's left ear, and a first speaker (255-1) positioned within the second temple (205) and thus positioned adjacent to the user's right ear.

[0067] The light-emitting module (not shown) may include at least one light-emitting element. The light-emitting module may emit light of a color corresponding to a specific state or emit light with an action corresponding to a specific state, in order to visually provide information regarding a specific state of the wearable device (103) to the user. For example, when the wearable device (103) requires charging, it may emit red light at a regular cycle. In one embodiment, the light-emitting module may be disposed on the first rim (201) and / or the second rim (202).

[0068] Referring to FIG. 2B, according to one embodiment, a wearable device (103) may include a printed circuit board (PCB) (290). The PCB (290) may be included in at least one of the first temple (204) or the second temple (205). The PCB (290) may include an interposer disposed between at least two sub-PCBs. One or more hardwares included in the wearable device (103) (e.g., hardwares to be described later in FIG. 5) may be disposed on the PCB (290). The wearable device (103) may include a flexible PCB (FPCB) for interconnecting the hardwares.

[0069] According to one embodiment, the wearable device (103) may include at least one of a gyro sensor, a gravity sensor, and / or an acceleration sensor for detecting a posture of the wearable device (103) and / or a posture of a body part (e.g., a head) of a user wearing the wearable device (103). Each of the gravity sensor and the acceleration sensor may measure gravitational acceleration and / or acceleration based on mutually perpendicular designated three-dimensional axes (e.g., an x-axis, a y-axis, and a z-axis). The gyro sensor may measure an angular velocity of each of the designated three-dimensional axes (e.g., an x-axis, a y-axis, and a z-axis). At least one of the gravity sensor, the acceleration sensor, and the gyro sensor may be referred to as an inertial measurement unit (IMU). According to one embodiment, the wearable device (103) may identify a user's motion and / or gesture performed to execute or cease a specific function of the wearable device (103) based on the IMU.

[0070] FIGS. 3A and 3B illustrate examples of the appearance of a wearable device according to various embodiments.

[0071] The wearable device (103) of FIGS. 3A and 3B may be an example of the electronic device (101) of FIG. 1. According to one embodiment, an example of the appearance of a first side (310) of a housing of the wearable device (103) may be illustrated in FIG. 3A, and an example of the appearance of a second side (320) opposite to the first side (310) may be illustrated in FIG. 3B.

[0072] Referring to FIG. 3A, according to one embodiment, a first surface (310) of a wearable device (103) may have a form attachable to a body part of a user (e.g., the face of the user). Although not shown, the wearable device (103) may further include a strap for fixing to a body part of a user, and / or one or more temples (e.g., the first temple (204) and / or the second temple (205) of FIGS. 2A and 2B). A first display (250-1) for outputting an image to a left eye among the user's two eyes, and a second display (250-2) for outputting an image to a right eye among the user's two eyes may be disposed on the first surface (310). The wearable device (103) is formed on the first surface (310) and may further include a rubber or silicone packing to prevent interference by light (e.g., ambient light) different from the light emitted from the first display (250-1) and the second display (250-2).

[0073] According to one embodiment, the wearable device (103) may include cameras (260-1) for photographing and / or tracking both eyes of the user adjacent to each of the first display (250-1) and the second display (250-2). The cameras (260-1) may be referred to as the gaze tracking camera (260-1) of FIG. 2B. According to one embodiment, the wearable device (103) may include cameras (260-5, 260-6) for photographing and / or recognizing the face of the user. The cameras (260-5, 260-6) may be referred to as FT cameras. The wearable device (103) may control an avatar representing the user in a virtual space based on the motion of the user's face identified using the cameras (260-5, 260-6).

[0074] Referring to FIG. 3b, a camera (e.g., cameras (260-7, 260-8, 260-9, 260-10, 260-11, 260-12)) and / or a sensor (e.g., a depth sensor (330)) for obtaining information related to the external environment of the wearable device (103) may be disposed on a second surface (320) opposite to the first surface (310) of FIG. 3a. For example, the cameras (260-7, 260-8, 260-9, 260-10) may be disposed on the second surface (320) to recognize external objects. Cameras (260-7, 260-8, 260-9, 260-10) may be referenced to the motion recognition cameras (260-2, 260-3) of FIG. 2B.

[0075] For example, using cameras (260-11, 260-12), the wearable device (103) can obtain images and / or videos to be transmitted to each of the user's eyes. The camera (260-11) can be placed on the second face (320) of the wearable device (103) to obtain an image to be displayed through the second display (250-2) corresponding to the right eye among the two eyes. The camera (260-12) can be placed on the second face (320) of the wearable device (103) to obtain an image to be displayed through the first display (250-1) corresponding to the left eye among the two eyes. The cameras (260-11, 260-12) can be referred to as the shooting camera (260-4) of FIG. 2B.

[0076] According to one embodiment, the wearable device (103) may include a depth sensor (330) disposed on the second face (320) to identify a distance between the wearable device (103) and an external object. Using the depth sensor (330), the wearable device (103) may obtain spatial information (e.g., a depth map) for at least a portion of the FoV of a user wearing the wearable device (103). Although not shown, a microphone may be disposed on the second face (320) of the wearable device (103) to obtain a sound output from an external object. The number of microphones may be one or more, depending on the embodiment.

[0077] FIG. 4 illustrates an example of a method for a wearable device to display information obtained from an external electronic device with respect to an external object.

[0078] FIG. 4 illustrates an example in which a wearable device (400) displays a visual object (415) in an area (425) of an external object (420) based on information obtained from an external electronic device (410). For example, the wearable device (400) of FIG. 4 may represent an example of the electronic device (101) of FIG. 1 or the wearable device (103) of FIGS. 2A to 3B.

[0079] Referring to FIG. 4, a wearable device (400) may be connected to an external electronic device (410). For example, the wearable device (400) may establish a connection with the external electronic device (410) through a communication circuit (e.g., the communication module (190) of FIG. 1). For example, the wearable device (400) may be connected to the external electronic device (410) through short-range communication. For example, the short-range communication may include BT (Bluetooth), BLE (Bluetooth low energy), Wi-Fi (Wireless fidelity), or UWB (ultra-wideband). However, the embodiments of the present disclosure are not limited thereto. For example, the wearable device (400) may be connected to the external electronic device (410) based on a wireless communication or wired communication technology that is different from the short-range communication.

[0080] According to one embodiment, the external electronic device (410) may include an electronic device for providing information for displaying a visual object to the wearable device (400). For example, the information may include information for displaying a screen associated with the external electronic device (410). For example, the visual object may include an image corresponding to the screen. For example, the visual object may be referred to as a virtual object, a graphical object, or a screen image. For example, the external electronic device (410) may represent a source that provides the information or an electronic device that can control the display of the visual object based on the information. For example, the external electronic device (410) may be referred to as a source electronic device, a source device, an external device, an external apparatus, or a first external electronic device. As described later in FIG. 5, the external electronic device (410) may include at least one of a user equipment, an electronic device (or wearable device) having the same account information as the wearable device (400), or a server that manages another external electronic device.

[0081] According to one embodiment, the wearable device (400) can identify an external object (420). For example, the wearable device (400) can identify an external object (420) located in a peripheral area of ​​the wearable device (400). For example, the peripheral area may include a field of view (FoV) of the wearable device (400). The external object (420) may include an object recognizable within the FoV. For example, the external object (420) may include a real object or a virtual object. For example, if the wearable device (400) provides an AR environment, the external object (420) may be a real object. Conversely, if the wearable device (400) provides a VR environment, the external object (420) may be a virtual object. According to one embodiment, the external object (420) may include an electronic device or a non-electronic device. For example, the external object (420) may include an electronic device that is connectable to the wearable device (400). Alternatively, for example, the external object (420) may include a non-electronic device that is not connectable to the wearable device (400), but that is recognizable by the wearable device (400). In one embodiment, when the external object (420) is an electronic device, the external object (420) that the wearable device (400) can recognize within the FoV may be the same as the external electronic device (410) that provides the information to the wearable device (400).

[0082] According to one embodiment, the wearable device (400) may identify an external object (420) from among one or more external objects recognizable within the FoV based on a specified event. The external object (420) identified from among the one or more external objects may be referred to as a target object. In addition, the external object (420) may be used as an area where a visual object (415) is displayed, and thus may be referred to as a display area or an expression area. For example, the specified event may include at least one of identifying that a first time that a user's gaze of the wearable device (400) is positioned within a specified range with respect to the external object (420) is greater than or equal to a first reference time, identifying that a second time that a body part (e.g., a hand) (403) of the user is positioned within the specified range is greater than or equal to a second reference time, or identifying a specified gesture with respect to the specified range. For example, the specified gesture may include a gesture, motion, or input (e.g., a double tap, etc.) set by the user. The specified event may be referred to as a recognition event, a recognition condition, a target object identification condition, or an identification condition. For example, the specified range may include an area extending from the periphery of the external object (420) by a specified length.

[0083] According to one embodiment, the wearable device (400) can identify one or more connectable external electronic devices. The one or more connectable external electronic devices may represent electronic devices that can be used as an area (425) where a visual object (415) is to be displayed, such as an external object (420). For example, the one or more connectable external electronic devices may be referred to as a second external electronic device. As described below, the wearable device (400) can identify the one or more connectable second external electronic devices through the communication circuit. For example, the one or more second external electronic devices may include at least one of an external electronic device connected to the same node (e.g., an access point (AP)) as the wearable device (400), an external electronic device whose signal strength received by the wearable device (400) is greater than or equal to a reference strength, or an external electronic device whose distance from the wearable device (400) identified based on a location positioning technique (e.g., Wi-Fi-based location positioning, BLE-based location positioning, IoT (internet of things) server-based location positioning, UWB (ultra-wideband)-based location positioning, ToF (time of flight) camera-based location positioning, GPS-based location positioning) is less than a reference distance. Additionally, for example, the one or more second external electronic devices may cause the wearable device (400) to identify the one or more second external electronic devices by providing a signal including information that identifies the wearable device (400) to a server (e.g., an IoT server) connected to the one or more second external electronic devices.

[0084] According to one embodiment, the wearable device (400) can identify an external electronic device (410) corresponding to an external object (420) identified based on the specified event. For example, the wearable device (400) can identify an external electronic device (410) corresponding to an attribute of the external object (420). Although not illustrated in FIG. 4, the wearable device (400) can be connected to one or more first external electronic devices including an external electronic device (410) that can operate as a source device. The wearable device (400) can identify the external electronic device (410) among the one or more first external electronic devices based on the attribute of the external object (420). For example, the wearable device (400) can identify the attribute of the external object (420) based on a camera or sensor included in the wearable device (400). Alternatively, for example, if the external object (420) is an electronic device, the wearable device (400) may identify the attribute based on information received from another electronic device (e.g., a server) connected to the external object (420) or from the external object (420). According to one embodiment, the wearable device (400) may identify the attribute of the external object (420) at the timing at which the external object (420) is identified. In addition, the wearable device (400) may identify the attribute of the external object (420) at a specified interval after the external object (420) is identified.

[0085] According to one embodiment, the property may include at least one of the size of the external object (420), the material (or surface material) of the external object (420), or the type (or category) of the external object (420). For example, the wearable device (400) may identify an external electronic device (410) that corresponds to the size of the external object (420) among the one or more first external electronic devices. Corresponding to the size may indicate that the difference between the size of the one or more first external electronic devices and the size of the external object (420) is minimal. In other words, among the one or more first external electronic devices, the external electronic device (410) may have a size that is most similar to the external object (420). However, the embodiments of the present disclosure are not limited thereto. For example, corresponding to the above size may mean that among the one or more first external electronic devices, the size and ratio of the external electronic device (410) may be most similar to the size and ratio of the external object (420). For example, the ratio may include the ratio between the horizontal length and the vertical length of the size of the external object (420).

[0086] According to one embodiment, the wearable device (400) may request information for displaying a visual object (415) from the external electronic device (410) based on another specified event. For example, the wearable device (400) may transmit a request signal for the information to the external electronic device (410) based on the other specified event. The wearable device (400) may receive a response signal for the request signal from the external electronic device (410). The response signal may include information for displaying the visual object (415). For example, the information for displaying the visual object (415) may include an image for a screen to be displayed on the external electronic device (410) or on another external electronic device connected to the external electronic device (410) (e.g., a server). In one embodiment, the designated other event may include at least one of: identifying that the first time is greater than or equal to a third reference time that is different from the first reference time; identifying that the second time is greater than or equal to a fourth reference time that is different from the second reference time; or identifying another designated gesture for the designated range. For example, the designated other gesture may be the same as the designated one. However, the embodiments of the present disclosure are not limited thereto. For example, the designated other gesture for requesting the information may be set differently from the designated gesture for recognizing an external object (420).

[0087] According to one embodiment, the wearable device (400) may generate a visual object (415) based on the information included in the response signal. For example, the wearable device (400) may generate a visual object (415) to be displayed through a display (e.g., the display module (160) of FIG. 1) based on the information. According to one embodiment, the wearable device (400) may display the generated visual object (415) within an area (425). Referring to example (401), the wearable device (400) may display a visual object (415) that at least partially overlaps an external object (420) within an area (425). For example, the area (425) may represent a display area of ​​the display corresponding to a location of the external object (420) within a real environment or a virtual environment. Example (401) illustrates a situation in which a wearable device (400) providing an AR environment recognizes a body part (403) of a user holding an external object (420), which is a note, and displays a visual object (415) within an area (425) of the external object (420). In example (401), the wearable device (400) may display the visual object (415) instead of the external object (420) held by the body part (403). Accordingly, the user may check information about a screen to be displayed by the external electronic device (410) based on the visual object (415) displayed through the area (425) of the external object (420).

[0088] Although not illustrated in FIG. 4, the wearable device (400) may, based on an input regarding the visual object (415), cease displaying the visual object (415) and display another visual object. Specific details related thereto are described below in FIGS. 10A and 10B.

[0089] A wearable device (400) according to an embodiment of the present disclosure can provide extended reality (XR). For example, the wearable device (400) can provide augmented reality (AR). Or, for example, the wearable device (400) can provide virtual reality (VR). For example, the wearable device (400) can recognize a body part (403) of the user (e.g., a hand or a finger) and perform control within an XR environment based on a movement or input of the body part (403). Or, for example, the wearable device (400) can perform control within the XR environment using a separate controller (e.g., an input device) that is different from the body part (403) of the user.

[0090] In a wearable device (400) providing an XR environment, when the user performs an input for a virtual object (e.g., a visual object (415) of FIG. 4), unlike a physical input, an input for a three-dimensional space may be performed, which may cause a sense of incongruity. In addition, when there are many virtual objects displayed within the FoV of the wearable device (400) as shown in example (401), selection and fine control of some virtual objects may be difficult. In addition, the criteria for selecting an external electronic device (e.g., an external electronic device (410)) among the one or more first external electronic devices and displaying information acquired from the selected external electronic device may not be clear. In addition, when the power level of the selected external electronic device is low, there may be restrictions on the display of virtual objects.

[0091] Hereinafter, an electronic device and a method according to embodiments of the present disclosure can recognize an external object (e.g., an external object (420) of FIG. 4) and display a virtual object (e.g., a visual object (415) of FIG. 4) on at least a portion of the recognized external object. The electronic device and the method according to embodiments of the present disclosure can recognize an external electronic device (e.g., an external electronic device (410) of FIG. 4) that provides information about the external object and the virtual object by having an electronic device (e.g., a wearable device (400)) that provides an XR environment identify a user's input. The electronic device and the method according to embodiments of the present disclosure can generate the virtual object for contents provided from the external electronic device and display the same. Accordingly, the electronic device and the method according to embodiments of the present disclosure can improve usability in an XR environment and provide an immersive user experience to the user by processing the user's input.

[0092] FIG. 5 illustrates an example of an external electronic device connected to a wearable device and an external object identified by the wearable device.

[0093] FIG. 5 illustrates an example (500) of a situation including a wearable device (400), external electronic devices (510, 520) connected to the wearable device (400), and external objects (540, 550, 560) identified by the wearable device (400). In the example (500), the wearable device (400) and two external electronic devices (510, 520) are connected, and three external objects (540, 550, 560) are identified. However, the embodiments of the present disclosure are not limited thereto. For example, the number of external electronic devices connected to the wearable device (400) may be one, three, or more. Furthermore, for example, the number of external objects identified by the wearable device (400) may be one, two, or four or more. The wearable device (400) of FIG. 5 may represent an example of the electronic device (101) of FIG. 1 or the wearable device (400) of FIGS. 2A to 3B.

[0094] Referring to FIG. 5, an exemplary situation is illustrated in which a wearable device (400), an external electronic device (510), and an external electronic device (520) are connected to each other based on a wired network and / or a wireless network. For example, the wired network may include a network such as the Internet, a local area network (LAN), a wide area network (WAN), or a combination thereof. For example, the wireless network may include a network such as long term evolution (LTE), 5g new radio (NR), wireless fidelity (Wi-Fi), Zigbee, near field communication (NFC), Bluetooth, Bluetooth low-energy (BLE), or a combination thereof. Although the wearable device (400), the external electronic device (510), and the external electronic device (520) are illustrated as being directly connected, the wearable device (400), the external electronic device (510), and the external electronic device (520) may be indirectly connected via one or more routers and / or access points (APs). In other words, the wearable device (400) is exemplified as being directly connected to a communication circuit (not shown) of an external electronic device (510) (or a communication circuit (not shown) of an external electronic device (520)) via a communication circuit (505), but the embodiments of the present disclosure are not limited thereto.

[0095] Referring to FIG. 5, according to one embodiment, a wearable device (400) may include at least one of a processor (501), a display (503), a communication circuit (505), and a memory (507). The processor (501), the display (503), the communication circuit (505), and the memory (507) may be electrically and / or operably coupled with each other by a communication bus. Hereinafter, operably coupled hardware components may mean that a direct connection or an indirect connection is established between the hardware components, either wired or wireless, such that a second hardware component is controlled by a first hardware component among the hardware components. Although illustrated based on different blocks, the embodiment is not limited thereto, and some of the hardware components illustrated in FIG. 5 (at least a portion of the processor (501), the display (503), the communication circuit (505), and the memory (507)) may be included in a single integrated circuit such as a system on a chip (SoC). The type and / or number of hardware components included in the wearable device (400) is not limited to those illustrated in FIG. 5. For example, the wearable device (400) may include only some of the hardware components illustrated in FIG. 5.

[0096] According to one embodiment, the processor (501) of the wearable device (400) may include a hardware component for processing data based on one or more instructions. The hardware component for processing data may include, for example, an arithmetic and logic unit (ALU), a floating point unit (FPU), and a field programmable gate array (FPGA). As an example, the hardware component for processing data may include a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processing (DSP), and / or a neural processing unit (NPU). The number of processors (501) may be one or more. For example, the processor (501) may have a multi-core processor structure such as a dual core, a quad core, or a hexa core. The processor (501) of FIG. 5 may include at least a part of the processor (120) of FIG. 1.

[0097] For example, the processor (501) may include various processing circuits and / or multiple processors. For example, the term "processor" as used herein, including in the claims, may include various processing circuits including at least one processor, one or more of which may be configured to individually and / or collectively perform the various functions described below in a distributed manner. As used herein, when "processor," "at least one processor," and "one or more processors" are described as being configured to perform various functions, these terms encompass, for example, and without limitation, situations where one processor performs some of the recited functions and other processor(s) perform other parts of the recited functions, and also situations where one processor may perform all of the recited functions. Additionally, the at least one processor may include a combination of processors that perform the various functions enumerated / disclosed, for example, in a distributed manner. At least one processor may execute program instructions to achieve or perform the various functions.

[0098] According to one embodiment, the display (503) of the wearable device (400) can output visualized information to the user (e.g., the visual object of FIG. 4, the visual object of FIG. 8, the visual object of FIG. 9, and the visual objects of FIGS. 10A and 10B). The number of displays (503) included in the wearable device (400) may be one or more. For example, the display (503) may be controlled by a processor (501) and / or a graphic processing unit (GPU) (not shown) to output visualized information to the user. The display (503) may include a flat panel display (FPD) and / or electronic paper. The FPD may include a liquid crystal display (LCD), a plasma display panel (PDP), a digital mirror device (DMD), one or more light emitting diodes (LEDs), and / or a micro LED. The above LED may include an OLED (organic LED). The display (503) of FIG. 5 may include at least a portion of the display module (160) of FIG. 1.

[0099] In one embodiment, light transmission may occur through at least a portion of the display (503). The wearable device (400) may provide a user experience related to augmented reality by providing a combination of light output through the display (503) and light transmitting through the display (503) to the user. As described above with reference to FIGS. 2A and 2B and / or 3A and 3B , the display (503) of the wearable device (400) according to one embodiment may have a structure for covering the entire FoV of the user, or radiating light toward the FoV of the user, while being worn on a body part of the user, such as the head. Although not illustrated, the wearable device (400) may include other output means for outputting information in forms other than visual or auditory forms. For example, the wearable device (400) may include at least one speaker for outputting an audio signal, and / or a motor (or actuator) for providing haptic feedback based on vibration. In the above example, a display (503) for providing a user experience related to the augmented reality is exemplified, but the embodiments of the present disclosure are not limited thereto. For example, the wearable device (400) may render an image for providing a user experience related to virtual reality, and display the rendered image through the display (503).

[0100] According to one embodiment, the communication circuit (505) of the wearable device (400) may include hardware for supporting transmission and / or reception of electrical signals between the wearable device (400) and the external electronic device (510) (or the external electronic device (520)). The communication circuit (505) may include, for example, at least one of a modem (MODEM), an antenna, and an optical / electronic (O / E) converter. The communication circuit (505) may support transmission and / or reception of electrical signals based on various types of communication means, such as Ethernet, Bluetooth (BT), Bluetooth low energy (BLE), ZigBee, long term evolution (LTE), 5G new radio (NR), and wireless-fidelity (WiFi). The communication circuit (505) of FIG. 5 may include at least a part of the communication module (190) and / or the antenna module (197) of FIG. 1.

[0101] According to one embodiment, the memory (509) of the wearable device (400) may include a hardware component for storing data and / or instructions input to and / or output from the processor (501). The memory (509) may include, for example, a volatile memory such as a random-access memory (RAM), and / or a non-volatile memory such as a read-only memory (ROM). The volatile memory may include, for example, at least one of a dynamic RAM (DRAM), a static RAM (SRAM), a cache RAM, and a pseudo SRAM (PSRAM). The non-volatile memory may include, for example, at least one of a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), a flash memory, a hard disk, a compact disc, and an embedded multimedia card (eMMC). The memory (509) of FIG. 5 may include at least a portion of the memory (130) of FIG. 1.

[0102] Although not illustrated in FIG. 5, the wearable device (400) according to one embodiment may include at least one sensor. For example, the wearable device (400) may include at least one sensor for identifying a designated event, another designated event, and a user input. For example, the at least one sensor may include an IMU (or IMU sensor), a bio sensor, an image sensor (or a camera), an acoustic sensor (or a microphone (or mic)), a gyro sensor, a gravity sensor, and / or an acceleration sensor. In addition, although not illustrated in FIG. 5, the wearable device (400) according to one embodiment may include an output means for outputting information in a form other than a visualized form. For example, the wearable device (400) may include a speaker for outputting an acoustic signal. For example, the wearable device (400) may include a motor for providing haptic feedback based on vibration.

[0103] Referring to FIG. 5, according to one embodiment, one or more instructions (or commands) representing operations and / or actions to be performed on data by a processor (501) of the wearable device (400) may be stored in the memory (509) of the wearable device (400). A set of one or more instructions may be referred to as a program, firmware, an operating system, a process, a routine, a sub-routine, and / or an application. Hereinafter, when an application is installed in an electronic device (e.g., a wearable device (400)), it may mean that one or more instructions provided in the form of an application are stored in the memory (509), and that the one or more applications are stored in a format executable by the processor of the electronic device (e.g., a file having an extension designated by the operating system of the wearable device (400)). According to one embodiment, the wearable device (400) may perform the operations of FIGS. 6 and 11 by executing one or more instructions stored in the memory (509).

[0104] According to one embodiment, the external electronic devices (510, 520) may represent source devices that provide information for displaying a virtual object (e.g., a visual object (400) of FIG. 4) to the wearable device (400). For example, the external electronic device (510) may be directly connected to the wearable device (400) and may provide the information for a screen to be displayed on the external electronic device (510). Alternatively, for example, the external electronic device (520) may be a server or electronic device connected to the external electronic device (530). For example, the external electronic device (520) may provide the information for a screen to be displayed on the external electronic device (530) to the wearable device (400). Although not illustrated in FIG. 5, for example, an external electronic device (520) may be connected to an external electronic device (510) and may provide the information obtained from the external electronic device (510) to the wearable device (400). For example, the external electronic devices (510, 520) may be referred to as a first external electronic device.

[0105] According to one embodiment, the wearable device (400) can identify external objects (540, 550, 560) within the FoV (509) of the wearable device (400). For example, when providing a VR environment, the wearable device (400) can identify external objects (540, 550, 560) that are virtual objects displayed within the FoV (509). Alternatively, for example, when providing an AR environment, the wearable device (400) can identify external objects (540, 550, 560) that are real objects existing within the FoV (509).

[0106] According to one embodiment, the wearable device (400) may identify a target object among external objects (540, 550, 560) based on a specified event. For example, the specified event may include at least one of identifying that a first time that a user's gaze of the wearable device (400) is positioned within a specified range with respect to the target object is greater than or equal to a first reference time, identifying that a second time that a body part (e.g., a hand) of the user is positioned within the specified range is greater than or equal to a second reference time, or identifying a specified gesture for the specified range. For example, the specified gesture may include a gesture, motion, or input (e.g., double tap, etc.) set by the user. The specified event may be referred to as a recognition event, a recognition condition, a target object identification condition, or an identification condition. For example, the specified range may include an area extending from a periphery of the target object by a specified length. For example, the wearable device (400) may identify the target object based on the specified event and display a menu for the target object. For example, the menu may include a visual object to guide selection of the target object. The wearable device (400) may select the target object based on an input to the menu.

[0107] In one embodiment, the wearable device (400) can identify one or more second external electronic devices that are connectable. For example, the one or more second external electronic devices that are connectable may represent electronic devices that can be used as external objects (540, 550, 560).

[0108] For example, the wearable device (400) can identify one or more second external electronic devices that are connectable through the communication circuit. For example, the one or more second external electronic devices can include at least one of an external electronic device connected to the same node (e.g., an access point (AP)) as the wearable device (400), an external electronic device whose signal strength received by the wearable device (400) is greater than or equal to a reference strength, or an external electronic device whose distance from the wearable device (400) identified based on a location positioning technique (e.g., Wi-Fi-based location positioning, BLE-based location positioning, IoT (internet of things) server-based location positioning) is less than a reference distance. Additionally, for example, the one or more second external electronic devices may cause the wearable device (400) to identify the one or more second external electronic devices by providing a signal including information that the one or more second external electronic devices have identified the wearable device (400) to a server (e.g., an IoT server) connected to the one or more second external electronic devices. According to one embodiment, the wearable device (400) may identify a third external electronic device located within the FoV (509) among the identified one or more second external electronic devices as an external object (e.g., external objects (540, 550, 560)).

[0109] According to one embodiment, among the external objects (540, 550, 560), the external object (560) may be identical to the external electronic device (510). In other words, the external object (560) may be an external electronic device (510) that is an external object identified within the FoV (509) and is a source device that provides the information. For example, the external object (560) may include a user terminal such as a smartphone or a tablet. Furthermore, according to one embodiment, among the external objects (540, 550, 560), the external object (550) may be identical to the external electronic device (530). In other words, the external object (550) may not be the first external electronic device directly connected to the wearable device (400), but may be an external electronic device (530) connected to the first external electronic device. Additionally, according to one embodiment, among the external objects (540, 550, 560), the external object (540) may be an object other than an electronic device. For example, the external object (540) may include a note or a mirror.

[0110] Figure 6 illustrates an example of an operational flow for a method in which a wearable device displays information obtained from an external electronic device on an external object.

[0111] At least some of the methods of FIG. 6 may be performed by the wearable device (400) of FIG. 5. For example, at least some of the methods may be controlled by the processor (501) of the wearable device (400).

[0112] Referring to FIG. 6, in operation (600), the wearable device (400) may connect with one or more first external electronic devices. For example, the wearable device (400) may establish a connection with the one or more first external electronic devices through the communication circuit (505). For example, the wearable device (400) may connect with the one or more first external electronic devices through short-range communication. For example, the short-range communication may include BT (Bluetooth), BLE (Bluetooth low energy), Wi-Fi (Wireless fidelity), or UWB (ultra-wideband). However, the embodiments of the present disclosure are not limited thereto. For example, the wearable device (400) may be connected to one or more first external electronic devices based on a wireless communication (e.g., a 4G network or a 5G network) or a wired communication technology that is different from the short-range communication.

[0113] According to one embodiment, the one or more first external electronic devices may include an electronic device for providing information for displaying a visual object to the wearable device (400). For example, the information may include information for displaying a screen associated with the one or more first external electronic devices. For example, the visual object may include an image corresponding to the screen. For example, the visual object may be referenced as a virtual object, a graphical object, or a screen image. For example, the one or more first external electronic devices may represent a source that provides the information or an electronic device that can control display of the visual object based on the information. For example, the one or more first external electronic devices may include at least one of a user equipment, an electronic device (or wearable device) having the same account information as the wearable device (400), or a server that manages another external electronic device.

[0114] In operation (605), the wearable device (400) may identify one or more second external electronic devices. For example, the wearable device (400) may identify the one or more second external electronic devices that are connectable via the communication circuit (505). For example, the one or more second external electronic devices may represent electronic devices that can be used (or identified) as external objects.

[0115] For example, the one or more second external electronic devices may include external electronic devices connected to the same node (e.g., AP (access point)) as the wearable device (400). For example, a plurality of APs may be deployed within a specific delay to provide a location-based service. The deployment of the plurality of APs may be configured according to the range of a service area or the coverage of the AP. In addition, a server (e.g., WPS (Wi-Fi Positioning System)) that manages APs may manage information on the plurality of APs and location information on the plurality of APs. For example, the wearable device (400) may identify the one or more second external electronic devices connected to the same AP as the wearable device (400) based on information on the plurality of APs and location information on the plurality of APs obtained from the server.

[0116] For example, the one or more second external electronic devices may include external electronic devices in which the strength of a signal received by the wearable device (400) is greater than or equal to a reference strength. For example, the wearable device (400) may estimate a distance based on the strength and pattern of the received signal for indoor positioning. For example, the strength of the signal may include a received signal strength indicator (RSSI). For example, the RSSI may be an indicator representing the strength of a signal (unit: dBm), in which the strength of the signal decreases with distance. The RSSI may be defined as in the following mathematical equation.

[0117]

[0118] The above n may represent an attenuation coefficient, the above A may represent an RSSI value measured at a specific location (e.g., 1 m (meter)) as a reference value, and the above d may represent a distance from a location where a signal is transmitted. For example, the attenuation constant may change depending on the environment in which the signal is transmitted and received. For example, the wearable device (400) may estimate a distance value based on the RSSI. The wearable device (400) may identify one or more second external electronic devices whose RSSI value is greater than or equal to a reference intensity. Alternatively, the wearable device (400) may identify one or more second external electronic devices whose estimated distance value is less than a reference distance.

[0119] For example, the one or more second external electronic devices may include external electronic devices whose distance from the wearable device (400) is less than a reference distance, as identified based on a positioning technique (e.g., Wi-Fi-based positioning, BLE-based positioning, IoT (internet of things) server-based positioning, UWB (ultra-wideband)-based positioning, ToF (time of flight) camera-based positioning, GPS-based positioning). For example, the wearable device (400) may identify distance information and direction information regarding the external electronic device through reception and transmission of high-frequency signals defined in 802.11.ad among Wi-Fi communication standards. Based on the distance information and the direction information, an AP connected to the external electronic device may identify an exact location of the electronic device. For example, the wearable device (400) may identify the one or more second external electronic devices whose distance, as identified based on the location acquired from the AP, is less than a reference distance. In addition, for example, the wearable device (400) can identify the location of the external electronic device based on the angle of arrival (AoA) and angle of departure (AoD) identified using BLE. For example, the wearable device can identify the one or more second external electronic devices, the distance identified based on the AoA and AoD being less than a reference distance. In addition, for example, the wearable device (400) can identify the one or more second external electronic devices based on information obtained from a server that manages IoT devices. For example, the server can pre-designate or identify the locations of external electronic devices, which are IoT devices connected to the server. For example, the wearable device (400) can identify the one or more second external electronic devices, the distance identified based on the server being less than a reference distance.For example, the one or more second external electronic devices may be identified based on a UWB sensor of the wearable device (400). For example, a UWB sensor (e.g., UWB radar or UWB ranging) may be used to identify the one or more second external electronic devices.

[0120] Additionally, for example, the one or more second external electronic devices may cause the wearable device (400) to identify the one or more second external electronic devices by providing a signal including information that the one or more second external electronic devices have identified the wearable device (400) to a server (e.g., an IoT server) connected to the one or more second external electronic devices. For example, the one or more second external electronic devices may identify a surrounding object based on a camera or a sensor. For example, a UWB sensor may be used to identify the object. In this case, the object may include the wearable device (400) identified based on the camera or the sensor. The one or more second external electronic devices may provide information about the identified wearable device (400) to the server. For example, the one or more second external electronic devices may display a user interface through a display. For example, the user interface may be a user interface for the user to confirm whether the device (e.g., wearable device (400)) to be identified through the server (or peer-to-peer (P2P)) is correct. Based on the user's input (or gesture) to the user interface, the one or more second external electronic devices may provide information to the server. The server may perform a connection (or mapping) between the wearable device (400) and the one or more second external electronic devices based on the information. At this time, the connection may be accompanied by authentication (e.g., a predetermined pattern) to enhance security. Accordingly, the wearable device (400) may also identify the one or more second external electronic devices. In the above example, the case where the object is the wearable device (400) is exemplified, but the embodiments of the present disclosure are not limited thereto.For example, by identifying a user of a given gesture or wearable device (400), the one or more second external electronic devices can identify the presence of the wearable device (400) and thereby provide information about the wearable device (400) to the server.

[0121] In operation (610), the wearable device (400) can identify one or more external objects within the FoV. For example, the FoV may be the FoV of the wearable device (400). For example, the wearable device (400) can identify one or more external objects within the FoV based on a camera, a sensor, or a communication circuit (505). At this time, the one or more external objects may be real objects. However, the embodiments of the present disclosure are not limited thereto. For example, when providing a virtual environment, the wearable device (400) may identify one or more external objects within the FoV among virtual objects included in the virtual environment.

[0122] According to one embodiment, the wearable device (400) may identify a third external electronic device located within the FoV among the identified one or more second external electronic devices as the one or more external objects. In other words, the one or more external objects may include the third external electronic device. Referring to the above, an example in which the wearable device (400) identifies the one or more second external electronic devices and uses the one or more second external electronic devices as one of the one or more external objects is described, but the embodiments of the present disclosure are not limited thereto. For example, the wearable device (400) may omit the operation (605) of identifying the one or more second external electronic devices and only identify the one or more external objects within the FoV. In this case, the one or more external objects may be objects other than electronic devices.

[0123] In operation (615), the wearable device (400) may identify a target object based on identifying a designated event. For example, the designated event may include at least one of identifying that a first time that a user's gaze of the wearable device (400) is positioned within a designated range for a specific external object among the one or more external objects is greater than or equal to a first reference time, identifying that a second time that a body part (e.g., a hand) of the user is positioned within the designated range is greater than or equal to a second reference time, or identifying a designated gesture for the designated range. For example, the designated gesture may include a gesture, motion, or input (e.g., double tap, etc.) set by the user. The designated event may be referred to as a recognition event, a recognition condition, a target object identification condition, or an identification condition. For example, the designated range may include an area extending from a periphery of the specific external object by a designated length.

[0124] According to one embodiment, the wearable device (400) may identify the target object based on identifying the designated event for the specific external object among the identified one or more external objects. For example, the target object may represent the specific external object.

[0125] In operation (620), the wearable device (400) may identify an external electronic device among the one or more first external electronic devices based on an attribute of the target object. For example, the wearable device (400) may identify the external electronic device based on the attribute. The external electronic device may represent a source device.

[0126] According to one embodiment, the wearable device (400) can identify the attribute. For example, the wearable device (400) can identify the attribute of the target object based on a camera or sensor included in the wearable device (400). Alternatively, for example, when the target object is an electronic device, the wearable device (400) can identify the attribute based on information received from another electronic device (e.g., a server) connected to the target object or from the target object. According to one embodiment, the wearable device (400) can identify the attribute of the target object at the timing at which the target object is identified. In addition, the wearable device (400) can identify the attribute of the target object at designated intervals after the target object is identified.

[0127] According to one embodiment, the property may include at least one of the size of the target object, the material (or surface material) of the target object, or the type (or category) of the target object. For example, the wearable device (400) may identify an external electronic device corresponding to the size of the target object among the one or more first external electronic devices. Corresponding to the size may indicate that the difference between the size of the one or more first external electronic devices and the size of the target object is minimal. In other words, among the one or more first external electronic devices, the external electronic device may have a size most similar to the target object.

[0128] For example, when the one or more first external electronic devices include only one first external electronic device, the wearable device (400) can identify the first external electronic device with respect to the target object. The wearable device (400) can transmit information about attributes (e.g., size) of the target object to the first external electronic device. The first external electronic device can identify execution of a specific function based on the information. For example, the specific function can include a function of changing a user interface for a display area identified according to information about attributes of the target object. For example, the specific function can include a DeX (desktop experience) function. For example, the wearable device (400) can transmit the information indicating a first size to the first external electronic device. It is assumed that the size of the display area of ​​the first external electronic device has a second size different from the first size. At this time, the first external electronic device can execute the specific function based on the information. Instead of generating an image for a screen corresponding to the second size, the first external electronic device can generate an image for a screen corresponding to the first size, and transmit information including the image to the wearable device (400) as described below. Alternatively, in one example, when the size of the display area of ​​the first external electronic device is the same as the first size, the specific function may include a mirroring function.

[0129] In operation (625), the wearable device (400) may receive information for displaying a screen associated with the external electronic device. For example, the wearable device (400) may receive the information for displaying the screen from the external electronic device. For example, the wearable device (400) may transmit a request signal for the information to the external electronic device based on another specified event. The wearable device (400) may receive a response signal for the request signal from the external electronic device. The response signal may include information for displaying the screen associated with the external electronic device. The screen associated with the external electronic device may include a screen to be displayed by the external electronic device or a screen to be displayed by another external electronic device connected to the external electronic device. In other words, the information for displaying the screen may include an image for a screen to be displayed by the external electronic device or another external electronic device connected to the external electronic device.

[0130] In one embodiment, the designated other event may include at least one of identifying that the first time is greater than or equal to a third reference time, identifying that the second time is greater than or equal to a fourth reference time, or identifying another designated gesture for the designated range. For example, the first time and the second time of the designated other event may represent a length of time from the time at which the target object is identified based on the designated event. For example, the designated other gesture may be identical to the designated gesture. However, the embodiments of the present disclosure are not limited thereto. For example, the designated other gesture for requesting the information may be set differently from the designated gesture for recognizing the target object.

[0131] The above example describes an embodiment in which the wearable device (400) obtains the information for displaying the screen from the external electronic device. However, the embodiments of the present disclosure are not limited thereto. For example, the wearable device (400) can identify data regarding the screen of the external electronic device stored by the wearable device (400). The visual object described below can be generated based on the data.

[0132] In operation (630), the wearable device (400) may display a visual object that at least partially superimposes on the target object. For example, the wearable device (400) may generate the visual object representing the screen based on the information received from the external electronic device. However, the embodiments of the present disclosure are not limited thereto. For example, the wearable device (400) may also receive the information including the visual object generated by the external electronic device.

[0133] According to one embodiment, the wearable device (400) may display the visual object within an area corresponding to the target object. For example, the area corresponding to the target object may represent a display area of ​​the display corresponding to the location of the target object within a real or virtual environment.

[0134] Although not illustrated in FIG. 6, the visual object displayed by the wearable device (400) may be generated based on information about a screen to be displayed by the external electronic device, obtained from the external electronic device. At this time, some functions of the external electronic device may be limited. For example, some of the functions may include a camera function or a music playback function executed by the external electronic device. For example, the photographing function of the camera function may be limited. Alternatively, the screen viewed by the external electronic device in the camera function may be displayed through the visual object displayed by the wearable device (400). In addition, for example, the music playback function may be executed by the wearable device (400) rather than the external electronic device. In other words, when the external electronic device plays a video, video information of the video to be displayed by the external electronic device may be displayed through the visual object on the wearable device (400). At this time, the audio information of the video can be output through the output device of the wearable device (400).

[0135] In the above example of FIG. 6, an embodiment in which the visual object is displayed in one of the above regions is described, but the embodiments of the present disclosure are not limited thereto. For example, the wearable device (400) may divide the region corresponding to the target object into a plurality of sub-regions. For example, the wearable device (400) may identify the region as a first sub-region and a second sub-region. The wearable device (400) may display a first visual object in the first sub-region and a second visual object in the second sub-region. For example, the wearable device (400) may receive information about a first screen and information about a second screen from the external electronic device, and generate the first visual object representing the first screen and the second visual object representing the second screen. Alternatively, the wearable device (400) may receive information about the first screen from the external electronic device (or the first source device) and receive information about the second screen from another external electronic device (or the second source device). Accordingly, the wearable device (400) may generate the first visual object representing the first screen and the second visual object representing the second screen.

[0136] In operation (635), the wearable device (400) may display another visual object based on identifying an event for a change in the visual object. For example, the event for the change may include obtaining an input for the visual object or changing an attribute of the target object.

[0137] According to one embodiment, the wearable device (400) may transmit information about the input to the external electronic device based on obtaining an input for the visual object. For example, the information about the input may include a relative position of the input with respect to the visual object. For example, the external electronic device may identify another screen that is changed from the screen based on the information about the input. For example, the external electronic device may generate input data for the screen to be displayed by the external electronic device based on the information about the input. The external electronic device may identify the other screen that is to be changed from the screen based on the input data. The external electronic device may transmit information for displaying the other screen to the wearable device (400). The wearable device (400) may generate the other visual object based on the information for displaying the other screen. The wearable device (400) can stop displaying the visual object and display another visual object in the area corresponding to the target object.

[0138] According to one embodiment, the wearable device (400) may identify one external electronic device (or source device) corresponding to the changed attribute among the one or more first external electronic devices based on the identification that the attribute of the target object has changed. For the convenience of the following description, the currently identified external electronic device is assumed to be the first source device.

[0139] For example, when the size of the target object changes from a first size to a second size, the wearable device (400) can identify a second source device having a size most similar to the changed second size. For example, the second source device can indicate a source device having different properties (e.g., size) from the first source device, which is the currently identified external electronic device. The wearable device (400) can request information for displaying another screen to be displayed on the second source device from the second source device. The wearable device (400) can receive the information for displaying the other screen received from the second source device and generate another visual object based thereon. The wearable device (400) can stop displaying the visual object and display the other visual object in the area corresponding to the target object.

[0140] Or, for example, when the size of the target object is changed from the first size to the second size, the wearable device (400) can transmit information about the changed second size to the first source device. The first source device can identify the execution of a specific function based on identifying the change to the second size. For example, the specific function can include a function of changing a user interface for a display area changed from the first size to the second size. For example, the specific function can include a DeX (desktop experience) function. The first source device can transmit information for displaying another screen to be displayed in the display area having the second size based on the execution of the specific function to the wearable device (400). The wearable device (400) can receive the information for displaying the other screen received from the first source device and generate another visual object based thereon. The wearable device (400) can stop displaying the visual object and display another visual object in the area corresponding to the target object.

[0141] Or, for example, when the size of the target object is changed from the first size to the second size, the wearable device (400) can transmit information about the changed second size to the first source device. At this time, the first source device may be an electronic device including a deformable flexible display including a folded state or an unfolded state. The first source device can identify that the change from the folded state to the unfolded state based on identifying that the change to the second size has occurred. In other words, the first source device can identify that the size of the display area has changed from the first size in the folded state to the second size in the unfolded state. At this time, the first source device can identify that the change to the second size has occurred when the first size is greater than or equal to a threshold size while changing from the folded state to the unfolded state. For example, the threshold size may be a size between the first size and the second size, and may indicate a change from the folded state to the unfolded state. Accordingly, the first source device may transmit information for displaying another screen to be displayed in a display area having the second size to the wearable device (400). The wearable device (400) may receive the information for displaying the other screen received from the first source device and generate another visual object based thereon. The wearable device (400) may stop displaying the visual object and display the other visual object in the area corresponding to the target object. In the above example, the first source device is described as an electronic device including a flexible display including the folded state and the unfolded state, but embodiments of the present disclosure are not limited thereto.For example, the first source device may include an electronic device including a flexible display having a slide-in state and a slide-out state.

[0142] In the embodiment of FIG. 6, an example is shown in which the wearable device (400) identifies one or more first external electronic devices (i.e., source devices) and one or more second external electronic devices that can be used as external objects that are electronic devices through the communication circuit (505), but the embodiments of the present disclosure are not limited thereto. For example, the wearable device (400) may omit the operation (600) and / or operation (605) of identifying through the communication circuit (505). For example, the wearable device (400) may identify one or more external objects within the FoV based on a camera, a sensor, or a communication circuit (e.g., UWB). The one or more external objects may be non-electronic devices. The wearable device (400) may identify the target object among the one or more external objects based on the specified event. The wearable device (400) can receive information for displaying a screen associated with the source device from the source device corresponding to the target object. Based on the information, the wearable device (400) can display a visual object that at least partially overlaps the target object.

[0143] FIGS. 7A through 7C illustrate examples of a method for identifying an external electronic device corresponding to a target object identified based on a specified event.

[0144] FIGS. 7A to 7C illustrate examples of a method for identifying an external electronic device corresponding to an attribute of a target object identified based on the wearable device (400) identifying the specified event.

[0145] For example, the specified event may include at least one of identifying that a first time that a user's gaze of the wearable device (400) is positioned within a specified range for a specific external object among the one or more external objects is greater than or equal to a first reference time, identifying that a second time that a body part (e.g., a hand) of the user is positioned within the specified range is greater than or equal to a second reference time, or identifying a specified gesture for the specified range. For example, the specified gesture may include a gesture, motion, or input (e.g., double tap, etc.) set by the user. The specified event may be referred to as a recognition event, a recognition condition, a target object identification condition, or an identification condition. For example, the specified range may include an area extending from a periphery of the specific external object by a specified length.

[0146] The target objects (710, 720, 730) and external electronic devices (740, 750, 760) illustrated in FIGS. 7A to 7C are merely examples for convenience of explanation, and the embodiments of the present disclosure are not limited thereto.

[0147] Referring to FIGS. 7A to 7C, the wearable device (400) can identify an external object held by the user's body part (403) (e.g., hand) as a target object. In the example of FIG. 7A, the wearable device (400) can identify a first external object (710) in contact with the body part (403) as the target object. In the example of FIG. 7B, the wearable device (400) can identify a second external object (720) in contact with the body part (403) as the target object. In this case, the second external object (720) may be an electronic device (e.g., a smartphone). In the example of FIG. 7C, the wearable device (400) can identify a third external object (730) in contact with the body part (403) as the target object.

[0148] According to one embodiment, the wearable device (400) may be connected to external electronic devices (740, 750, 760). For example, the wearable device (400) may establish a connection with the external electronic devices (740, 750, 760) through the communication circuit (505). For example, the wearable device (400) may be directly connected to the external electronic devices (740, 750, 760) through the communication circuit (505), or may be indirectly connected to the external electronic devices (740, 750, 760) through another external electronic device (e.g., a server). Each of the external electronic devices (740, 750, 760) may be referred to as a first external electronic device (or source device).

[0149] According to one embodiment, the wearable device (400) can identify an external electronic device corresponding to an attribute of the target object. For convenience of explanation, examples of the attribute indicating the size of the target object are described in FIGS. 7A to 7C , but the embodiments of the present disclosure are not limited thereto. In the example of FIG. 7A , the wearable device (400) can identify an external electronic device (740) corresponding to a first external object (710). In the example of FIG. 7B , the wearable device (400) can identify an external electronic device (740) corresponding to a second external object (720). In this case, the second external object (720) can represent the same electronic device as the external electronic device (740). In the example of FIG. 7C , the wearable device (400) can identify an external electronic device (750) corresponding to a third external object (730). As mentioned above, the external electronic device (760) may not be identified. The arrows (or indicators) in FIGS. 7A to 7C are only used to visually indicate a correspondence between an external object and an identified external electronic device, and do not indicate that a communication connection is established between the external object and the identified external electronic device.

[0150] Referring to FIGS. 7A to 7C , the wearable device (400) can identify an external electronic device (or the first external electronic device, the source device) based on the attributes of the target object. Obtaining information from the identified external electronic device and displaying a visual object based on the acquired information is described below in FIG. 8 .

[0151] FIG. 8 illustrates examples of a method for displaying a visual object generated based on information obtained from an external electronic device on an area of ​​an external object.

[0152] FIG. 8 illustrates examples (800, 850) of how a wearable device (400) displays a visual object generated on the area based on information obtained from an external electronic device (740) identified for a first external object (710) of FIG. 7a and a second external object (720) of FIG. 7c.

[0153] Example (800) illustrates a case where a visual object (820) is displayed on an area (810) based on the information obtained from an external electronic device (740) identified for a first external object (710), which is a target object. In example (800) of FIG. 8, a visual object (820) including an appearance of an external electronic device (740) and a screen displayed on the external electronic device (740) is exemplified, but the embodiments of the present disclosure are not limited thereto. For example, the visual object (820) may include the appearance and the screen among the screens. For example, the wearable device (400) may identify a first external object (710) held by a user's body part (403) as a target object, and identify the external electronic device (740) based on an attribute (e.g., size) of the first external object (710). For example, the wearable device (400) may receive, from the external electronic device (740), the information for displaying a screen to be displayed by the external electronic device (740). For example, the wearable device (400) may generate a visual object (820) based on the information. The wearable device (400) may display the visual object (820) in the area (810).

[0154] Example (850) illustrates a case where a visual object (870) is displayed on an area (860) based on the information obtained from an identified external electronic device (740) for a second external object (720), which is a target object. In example (850) of FIG. 8, a visual object (870) including the appearance of the external electronic device (740) and a screen displayed on the external electronic device (740) is exemplified, but the embodiments of the present disclosure are not limited thereto. For example, the visual object (870) may include the appearance and the screen among the screens. In this case, the second external object (720) may be identical to the external electronic device (740). In this case, the wearable device (400) may receive the information about the screen to be displayed when the display of the external electronic device (740) is turned on. For example, the wearable device (400) may generate a visual object (870) based on the above information. The visual object (870) may correspond to the screen to be displayed when the display of the external electronic device (740) is turned on. The wearable device (400) may display the visual object (870) on the area (860). At this time, while the visual object (870) is displayed, the display of the external electronic device (740) located in the actual space may be turned off.

[0155] Figure 9 illustrates an example of a method for displaying visual objects on multiple subregions of an external object.

[0156] FIG. 9 illustrates examples (900, 905) of how a wearable device (400) displays visual objects (931, 932) on multiple subareas (921, 922) of an external object (710).

[0157] Example (900) illustrates a case where visual objects (931, 932) are displayed on a plurality of partial regions (921, 922) based on information obtained from an identified external electronic device (not shown) regarding an external object (910) that is a target object. For example, a wearable device (400) may identify an external object (910) held by a user's body part (403) as a target object, and identify the external electronic device based on the size of the external object (910). At this time, the external electronic device may include one source device or two source devices.

[0158] For example, when the external electronic device is one source device, the wearable device (400) may receive information about a first screen and information about a second screen from the source device, and generate a first visual object (931) representing the first screen and a second visual object (932) representing the second screen. For example, the first screen and the second screen may represent multiple screens provided by one software application (e.g., an Internet browser). Or, for example, the first screen may represent a screen provided by a first software application, and the second screen may represent a screen provided by a second software application different from the first software application (e.g., a multi-window). Or, for example, the first screen may represent a home screen, and the second screen may represent a screen provided by a software application executed based on an input to the home screen (e.g., a multi-task screen). Referring to example (905), the wearable device (400) can display a first visual object (931) in a first partial area (921) and a second visual object (932) in a second partial area (922).

[0159] Alternatively, for example, if the external electronic devices are two source devices (e.g., a first source device and a second source device), the wearable device (400) can receive information about a first screen from the first source device and information about a second screen from the second source device. The wearable device (400) can generate a first visual object (931) representing the first screen and a second visual object (932) representing the second screen. Referring to example (905), the wearable device (400) can display the first visual object (931) in the first partial area (921) and can display the second visual object (932) in the second partial area (922).

[0160] Figure 10a illustrates an example of how to change the visual objects displayed based on user input.

[0161] FIG. 10A illustrates examples (1001, 1002, 1003) of a method for changing a visual object (1020) displayed in an area (1015) based on information obtained from an external electronic device (not shown) corresponding to an external object (1010) to another visual object (1030) based on a user input, in a state where the wearable device (400) displays a visual object (1020) based on information obtained from an external electronic device (not shown) corresponding to an external object (1010).

[0162] Referring to example (1001), the wearable device (400) can identify an external object (1010) as a target object. For example, the wearable device (400) can identify an external object (1010) held by a body part (403) of the user as a target object, and identify the external electronic device based on the size of the external object (1010).

[0163] Referring to example (1002), the wearable device (400) may generate a visual object (1020) based on the information acquired from the external electronic device. For example, the visual object (1020) may correspond to a screen associated with the external electronic device generated based on the information. For example, the visual object (1020) may include a home screen of the external electronic device. The wearable device (400) may display the visual object (1020) in an area (1015) corresponding to the external object (1010). While displaying the visual object (1020), the wearable device (400) may obtain an input (1025). For example, the input (1025) may include a touch input of the user to a portion of the visual object (1020). For example, the portion of the area may be an area where a music playback application is located within the home screen. The wearable device (400) may transmit information about the input (1025) to the external electronic device based on acquiring an input (1025) for a visual object (1020). For example, the information about the input (1025) may include a relative position of the input (1025) with respect to the visual object (1020). For example, the external electronic device may identify another screen that is changed from the screen based on the information about the input (1025). For example, the other screen may include a user interface of the music playback application. For example, the external electronic device may generate input data for the screen to be displayed by the external electronic device based on the information about the input (1025). The external electronic device may identify the other screen that is to be changed from the screen based on the input data. The external electronic device may transmit information for displaying the other screen to the wearable device (400).

[0164] Referring to example (1003), the wearable device (400) may display another visual object (1030) generated based on information for displaying the other screen. For example, the other visual object (1030) may correspond to the other screen associated with the external electronic device generated based on the information for displaying the other screen. The wearable device (400) may stop displaying the visual object (1020) and display another visual object (1030). For example, the other visual object (1030) may include the user interface of the music playback application.

[0165] In the above example, an embodiment is described in which the wearable device (400) requests information for generating another visual object (1030) based on an input (1025) for a visual object (1020) (i.e., an input for a virtual space), but the embodiments of the present disclosure are not limited thereto. For example, as in the example (850) of FIG. 8, when the external object (1010) as a target object and the external electronic device as a source device are the same, the user may perform a touch input on the display of the external electronic device. For example, the external electronic device may identify the touch input based on a sensor (e.g., a touch sensor) when the display of the external electronic device is turned off. The external electronic device may generate information for displaying the other screen based on the touch input. The wearable device (400) may receive information for displaying the other screen from the external electronic device and generate another visual object (1030). Accordingly, the wearable device (400) can stop displaying the visual object (1020) and display another visual object (1030).

[0166] Figure 10b illustrates an example of how to change the displayed visual object based on a change in the size of an external object.

[0167] FIG. 10B illustrates examples (1005, 1006) in which a wearable device (400) displays a first visual object (1060) in a first area (1055-1) based on information acquired from an external electronic device (hereinafter, a first source device) (not shown) corresponding to an external object (1050-1). In addition, FIG. 10B illustrates examples (1007, 1008) in which a wearable device (400) displays a second visual object (1070) in a second area (1055-2) as a property of the external object (1050-1) changes while displaying the first visual object (1060) in the first area (1055-1) as in example (1006).

[0168] Referring to example (1005), the wearable device (400) can identify an external object (1050-1) held by the user's body part (403) as a target object, and identify the first source device based on the size of the external object (1050-1). The wearable device (400) can obtain information for generating a first visual object (1060) from the first source device (or information for displaying a screen associated with the first source device). Referring to example (1006), the wearable device (400) can generate a first visual object (1060) based on the information, and display it in the first area (1055-1).

[0169] Referring to example (1007), in a state where the first visual object (1060) is displayed in the first area (1055-1) as in example (1006), the wearable device (400) can identify that the property of the external object (1050-1) has changed. For example, the wearable device (400) can identify that the external object (1050-1) has changed to the external object (1050-2). For example, if the external object (1050-1) is a note, the change of the external object (1050-1) to the external object (1050-2) may include the note changing from a closed state to an unfolded state. Accordingly, the wearable device (400) can identify that the property (e.g., size) of the external object (1050-1) has changed. The wearable device (400) can identify a second source device corresponding to the target object, the external object (1050-2), based on the change in the above property. For example, the second source device can have a wider display area than the first source device. The wearable device (400) can obtain information for generating a second visual object (1070) (or information for displaying a screen associated with the second source device) from the second source device. Referring to example (1008), the wearable device (400) can stop displaying the first visual object (1060) and display the second visual object (1070) generated based on the information in the second area (1055-2).

[0170] In the above-described example, an embodiment is described in which a second visual object (1070) is displayed based on information of the second source device changed from the first source device as the size of the external object (1050-1) changes to the size of the external object (1050-2), but the embodiment of the present disclosure is not limited thereto. For example, as the size of the external object (1050-1) changes to the size of the external object (1050-2), the wearable device (101) may receive information for displaying the second visual object (1070) from the first source device.

[0171] For example, when the external object (1050-1) changes to the external object (1050-2), the wearable device (400) can transmit information about the size of the external object (1050-2) to the first source device. The first source device can identify the execution of a specific function based on identifying the changed size. For example, the specific function can include a function of changing a user interface for a display area having the changed size. For example, the specific function can include a DeX (desktop experience) function or a mirroring function. The first source device can transmit information for displaying another screen to be displayed in the display area having the changed size according to the execution of the specific function to the wearable device (400). The wearable device (400) can receive information for displaying the other screen received from the first source device and generate a second visual object (1070) based on the information. The wearable device (400) can stop displaying the first visual object (1060) and display the second visual object (1070) in the second area (1055-2).

[0172] Or, for example, when changing from an external object (1050-1) to an external object (1050-2), the wearable device (400) can transmit information about the size of the external object (1050-2) to the first source device. For example, in the case of external objects (1050-1, 1050-2) that are notes, the wearable device (400) can identify the unfolded state (e.g., unfolded angle and unfolded degree) when changing from the external object (1050-1) to the external object (1050-2) and provide information about the unfolded state to the first source device. At this time, the first source device can be an electronic device including a flexible display that is deformable into a folded state or an unfolded state. The first source device can identify that the folded state has changed to the unfolded state based on identifying the changed size (or information about the unfolded state). In other words, the first source device can identify that the size of the display area has changed even if the state of the flexible display has not actually changed. Accordingly, the first source device can transmit information for displaying another screen to be displayed in the display area having the changed size to the wearable device (400). The wearable device (400) can receive the information for displaying the other screen received from the first source device and generate a second visual object (1070) based thereon. The wearable device (400) can stop displaying the first visual object (1060) and display the second visual object (1070) in the second area (1055-2). In the above-described example, the first source device is described as an electronic device including a flexible display including the folded state and the unfolded state, but the embodiments of the present disclosure are not limited thereto.For example, the first source device may include an electronic device including a flexible display having a slide-in state and a slide-out state.

[0173] FIG. 11 illustrates an example of an operational flow for a method in which a wearable device displays a visual object generated based on information acquired from an external electronic device on an external object.

[0174] At least some of the above methods of FIG. 11 may be performed by the wearable device (400) of FIG. 5. For example, at least some of the above methods may be controlled by the processor (501) of the wearable device (400).

[0175] Referring to FIG. 11, in operation (1110), the wearable device (400) may be connected to one or more external electronic devices. For example, the wearable device (400) may establish a connection with the one or more external electronic devices through the communication circuit (505). For example, the wearable device (400) may be connected to the one or more external electronic devices through short-range communication. For example, the short-range communication may include Bluetooth (BT), Bluetooth low energy (BLE), Wireless fidelity (Wi-Fi), or Ultra-Wideband (UWB). However, the embodiments of the present disclosure are not limited thereto. For example, the wearable device (400) may be connected to the one or more external electronic devices based on a wireless communication (e.g., a 4G network or a 5G network) or a wired communication technology that is different from the short-range communication. The one or more external electronic devices may be referred to as one or more first external electronic devices.

[0176] According to one embodiment, the one or more external electronic devices may include an electronic device for providing information for displaying a visual object to the wearable device (400). For example, the information may include information for displaying a screen associated with the one or more external electronic devices. For example, the visual object may include an image corresponding to the screen. For example, the visual object may be referenced as a virtual object, a graphical object, or a screen image. For example, the one or more external electronic devices may represent a source that provides the information or an electronic device that can control display of the visual object based on the information. For example, the one or more external electronic devices may include at least one of a user equipment, an electronic device (or wearable device) having the same account information as the wearable device (400), or a server that manages another external electronic device.

[0177] In operation (1120), the wearable device (400) may identify a target object among one or more external objects within the FoV of the wearable device (400). For example, the wearable device (400) may identify one or more external objects within the FoV based on a camera, a sensor, or a communication circuit (505). At this time, the one or more external objects may be real objects. However, the embodiments of the present disclosure are not limited thereto. For example, when providing a virtual environment, the wearable device (400) may also identify one or more external objects within the FoV among virtual objects included in the virtual environment.

[0178] Although not illustrated in FIG. 11, according to one embodiment, the wearable device (400) can identify one or more second external electronic devices that are connectable through the communication circuit. For example, the one or more second external electronic devices may include at least one of an external electronic device connected to the same node (e.g., an access point (AP)) as the wearable device (400), an external electronic device whose signal strength received by the wearable device (400) is greater than or equal to a reference strength, or an external electronic device whose distance from the wearable device (400) identified based on a location positioning technique (e.g., Wi-Fi-based location positioning, BLE-based location positioning, IoT (internet of things) server-based location positioning, UWB (ultra-wideband)-based location positioning, ToF (time of flight) camera-based location positioning, GPS-based location positioning) is less than a reference distance. Additionally, for example, the one or more second external electronic devices may cause the wearable device (400) to identify the one or more second external electronic devices by providing a signal including information identifying the wearable device (400) to a server (e.g., an IoT server) connected to the one or more second external electronic devices. As for specific details related thereto, the contents of operation (605) of FIG. 6 may be substantially equally applied.

[0179] According to one embodiment, the wearable device (400) may identify a third external electronic device located within the FoV among the identified one or more second external electronic devices as the one or more external objects. In other words, the one or more external objects may include the third external electronic device.

[0180] According to one embodiment, the wearable device (400) may identify the target object based on identifying a designated event. For example, the designated event may include at least one of identifying that a first time period during which a user's gaze of the wearable device (400) is positioned within a designated range for a specific external object among the one or more external objects is greater than or equal to a first reference time period, identifying that a second time period during which a body part (e.g., a hand) of the user is positioned within the designated range is greater than or equal to a second reference time period, or identifying a designated gesture for the designated range. For example, the designated gesture may include a gesture, motion, or input (e.g., a double tap, etc.) set by the user. The designated event may be referred to as a recognition event, a recognition condition, a target object identification condition, or an identification condition. For example, the designated range may include an area extending from a periphery of the specific external object by a designated length. For example, the wearable device (400) may identify the target object based on the specified event and display a menu for the target object. For example, the menu may include a visual object to guide selection of the target object. The wearable device (400) may select the target object based on an input to the menu.

[0181] According to one embodiment, the wearable device (400) may identify the target object based on identifying the designated event for the specific external object among the identified one or more external objects. For example, the target object may represent the specific external object.

[0182] In operation (1130), the wearable device (400) may identify an external electronic device among the one or more external electronic devices based on the size of the target object. For example, the wearable device (400) may identify an external electronic device among the one or more first external electronic devices that corresponds to the size of the target object. Corresponding to the size may indicate that the difference between the size of the one or more first external electronic devices and the size of the target object is minimal. In other words, among the one or more first external electronic devices, the external electronic device may have a size that is most similar to the target object.

[0183] In FIG. 11, an example of identifying the external electronic device based on the size of the target object is described, but the embodiments of the present disclosure are not limited thereto. For example, the wearable device (400) may identify the external electronic device based on an attribute of the target object. For example, the attribute may include at least one of the size of the target object, the material (or surface material) of the target object, or the type (or category) of the target object.

[0184] In operation (1140), the wearable device (400) may receive information for displaying a screen associated with the external electronic device. For example, the wearable device (400) may receive the information for displaying the screen from the external electronic device. For example, the wearable device (400) may transmit a request signal for the information to the external electronic device based on another specified event. The wearable device (400) may receive a response signal to the request signal from the external electronic device. The response signal may include information for displaying the screen associated with the external electronic device. The screen associated with the external electronic device may include a screen to be displayed by the external electronic device or a screen to be displayed by another external electronic device connected to the external electronic device. In other words, the information for displaying the screen may include an image for a screen to be displayed by the external electronic device or another external electronic device connected to the external electronic device.

[0185] In one embodiment, the designated other event may include at least one of: identifying that the first time is greater than or equal to a third reference time that is different from the first reference time; identifying that the second time is greater than or equal to a fourth reference time that is different from the second reference time; or identifying another designated gesture for the designated range. For example, the designated other gesture may be the same as the designated one. However, the embodiments of the present disclosure are not limited thereto. For example, the designated other gesture for requesting the information may be set differently from the designated gesture for recognizing the target object.

[0186] In operation (1150), the wearable device (400) may display a visual object that at least partially superimposes on the target object. For example, the wearable device (400) may generate the visual object representing the screen based on the information received from the external electronic device. However, the embodiments of the present disclosure are not limited thereto. For example, the wearable device (400) may also receive the information including the visual object generated by the external electronic device.

[0187] Although not illustrated in FIG. 11, the visual object displayed by the wearable device (400) may be generated based on information about a screen to be displayed by the external electronic device, obtained from the external electronic device. At this time, some functions of the external electronic device may be limited. For example, some of the functions may include a camera function or a music playback function executed by the external electronic device. For example, the photographing function of the camera function may be limited. Alternatively, the screen viewed by the external electronic device in the camera function may be displayed through the visual object displayed by the wearable device (400). In addition, for example, the music playback function may be executed by the wearable device (400) rather than the external electronic device. In other words, when the external electronic device plays a video, video information of the video to be displayed by the external electronic device may be displayed through the visual object on the wearable device (400). At this time, the audio information of the video can be output through the output device of the wearable device (400).

[0188] According to one embodiment, the wearable device (400) may display the visual object within an area corresponding to the target object. For example, the area corresponding to the target object may represent a display area of ​​the display corresponding to the location of the target object within a real or virtual environment.

[0189] In the embodiment of FIG. 11, an example is shown in which the wearable device (400) identifies one or more first external electronic devices (i.e., source devices) and one or more second external electronic devices that can be used as external objects that are electronic devices through the communication circuit (505), but the embodiment of the present disclosure is not limited thereto. For example, the wearable device (400) may omit the operation (1110) of identifying through the communication circuit (505). For example, the wearable device (400) may identify one or more external objects within the FoV based on a camera, a sensor, or a communication circuit (e.g., UWB). The one or more external objects may be non-electronic devices. The wearable device (400) may identify the target object among the one or more external objects based on the specified event. The wearable device (400) can receive information for displaying a screen associated with the source device from the source device corresponding to the target object. Based on the information, the wearable device (400) can display a visual object that at least partially overlaps the target object.

[0190] As described above, the wearable device (400) may include a memory (507) that stores instructions and includes one or more storage media. The wearable device (400) may include a display (503). The wearable device (400) may include a communication circuit (505). The wearable device (400) may include at least one processor (501) that includes a processing circuit. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (400) to connect with one or more external electronic devices via the communication circuit (505). The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (400) to identify a target object among one or more external objects within a field of view (FoV) of the wearable device (400) based on identifying a designated event. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (400) to identify an external electronic device among the one or more external electronic devices based on an attribute of the target object. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (400) to receive, from the external electronic device, information for displaying a screen associated with the external electronic device.The above instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (400) to display, through the display (503), a visual object generated based on the information and at least partially superimposed on the target object.

[0191] According to one embodiment, the one or more external electronic devices may include at least one of an electronic device connected to the wearable device (400) via short-range communication, an electronic device including the same account information as the wearable device (400), or a server connected to another external electronic device and managing the other external electronic device.

[0192] According to one embodiment, when the external electronic device is the server, the information for displaying the screen may include an image for the screen to be displayed on the other external electronic device.

[0193] According to one embodiment, the specified event may include at least one of a first time during which a gaze of a user of the wearable device (400) is positioned within a specified range with respect to the target object for a first reference time or longer, a second time during which a body part of the user is positioned within the specified range for a second reference time or longer, or a specified gesture for the specified range.

[0194] According to one embodiment, the one or more external electronic devices may be one or more first external electronic devices. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (400) to identify one or more second external electronic devices via the communication circuit (505). The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (400) to identify a third external electronic device within the FoV among the one or more second external electronic devices. The instructions, when individually or collectively executed by the at least one processor (501), may be configured to cause the wearable device (400) to identify the target object among the one or more external objects including the third external electronic device.

[0195] According to one embodiment, the one or more second external electronic devices may include at least one of an external electronic device connected to an access point (AP) to which the wearable device (400) is connected, an external electronic device having a signal intensity received by the wearable device (400) that is equal to or greater than a reference intensity, or an external electronic device having a distance from the wearable device (400) identified based on a positioning technique that is less than a reference distance.

[0196] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (400) to receive a signal from a server connected to the one or more second external electronic devices, the signal indicating the one or more second external electronic devices. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (400) to identify the one or more second external electronic devices based on the signal. The signal may be transmitted from the server in response to another signal indicating that the one or more second external electronic devices identify the wearable device (400).

[0197] According to one embodiment, the identified external electronic device among the one or more external electronic devices may be the third external electronic device. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (400) to identify the third external electronic device among the one or more external objects including the third external electronic device as the target object. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (400) to receive the information from the third external electronic device. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (400) to display the visual object through the display (503) in a state where another display of the third external electronic device is turned off.

[0198] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (400) to receive, from the third external electronic device, information for displaying another screen to be changed from the screen. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (400) to cease displaying the visual object. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (400) to display, through the display (503), another visual object generated based on the information for displaying the another screen and at least partially overlapping with the target object. Information for displaying the other screen may be generated based on a touch input to the other display while the other display is turned off.

[0199] According to one embodiment, the attribute may include at least one of the size of the target object, the material of the target object, or the type of the target object. If the attribute is the size, the external electronic device may indicate an external electronic device among the one or more external electronic devices that has the smallest difference between the size of the display area and the size of the target object.

[0200] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (400) to transmit a request signal for the information to the external electronic device based on identifying another designated event. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (400) to receive, from the external electronic device, a response signal to the request signal including the information.

[0201] According to one embodiment, the one or more external electronic devices may include a plurality of external electronic devices. The plurality of external electronic devices may include the external electronic device and another external electronic device. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (400) to display the visual object on a first partial region of the target object. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (400) to display another visual object on a second partial region different from the first partial region of the target object. The another visual object may correspond to another screen to be displayed on the other external electronic device.

[0202] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (400) to obtain input for a portion of the visual object. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (400) to transmit information about the portion of the area to the external electronic device. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (400) to receive, from the external electronic device, information for displaying another screen to be changed from the screen based on the information about the portion of the area. The above instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (400) to cease displaying the visual object. The above instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (400) to display, through the display (503), another visual object generated based on information for displaying the other screen and at least partially overlapping with the target object.

[0203] According to one embodiment, the one or more external electronic devices may include a plurality of external electronic devices. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (400) to identify that the size of the target object has changed from a first size to a second size, if the attribute is the size of the target object. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (400) to identify, based on the second size, another external electronic device among the plurality of external electronic devices that corresponds to the second size. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (400) to receive, from the another external electronic device, information for displaying another screen to be displayed on the another external electronic device. The above instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (400) to cease displaying the visual object. The above instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (400) to display, through the display (503), another visual object generated based on information for displaying the other screen and at least partially overlapping with the target object.

[0204] As described above, the method performed by the wearable device (400) may include an operation of connecting with one or more external electronic devices. The method may include an operation of identifying a target object among one or more external objects within a field of view (FoV) of the wearable device (400) based on identifying a designated event. The method may include an operation of identifying an external electronic device among the one or more external electronic devices based on an attribute of the target object. The method may include an operation of receiving, from the external electronic device, information for displaying a screen associated with the external electronic device. The method may include an operation of displaying a visual object generated based on the information and at least partially superimposed on the target object.

[0205] According to one embodiment, the one or more external electronic devices may include at least one of an electronic device connected to the wearable device (400) via short-range communication, or a server connected to another external electronic device and managing the other external electronic device.

[0206] According to one embodiment, when the external electronic device is the server, the information for displaying the screen may include an image for the screen to be displayed on the other external electronic device.

[0207] According to one embodiment, the specified event may include at least one of a first time during which a gaze of a user of the wearable device (400) is positioned within a specified range with respect to the target object for a first reference time or longer, a second time during which a body part of the user is positioned within the specified range for a second reference time or longer, or a specified gesture for the specified range.

[0208] According to one embodiment, the one or more external electronic devices may be one or more first external electronic devices. The method may include an operation of identifying one or more second external electronic devices. The method may include an operation of identifying a third external electronic device within the FoV among the one or more second external electronic devices. The method may include an operation of identifying the target object among the one or more external objects including the third external electronic device.

[0209] According to one embodiment, the one or more second external electronic devices may include at least one of an external electronic device connected to an access point (AP) to which the wearable device (400) is connected, an external electronic device having a signal intensity received by the wearable device (400) that is equal to or greater than a reference intensity, or an external electronic device having a distance from the wearable device (400) identified based on a positioning technique that is less than a reference distance.

[0210] As described above, a non-transitory computer-readable storage medium may store one or more programs including instructions that, when individually or collectively executed by at least one processor (501) of a wearable device (400) including a display (503) and a communication circuit (505), cause the wearable device (400) to connect with one or more external electronic devices via the communication circuit (505). The non-transitory computer-readable storage medium may store one or more programs including instructions that, when individually or collectively executed by the at least one processor (501), cause the wearable device (400) to identify a target object among one or more external objects within a field of view (FoV) of the wearable device (400) based on identifying a designated event. The non-transitory computer-readable storage medium may store one or more programs including instructions that, when individually or collectively executed by the at least one processor (501), cause the wearable device (400) to identify an external electronic device among the one or more external electronic devices based on an attribute of the target object. The non-transitory computer-readable storage medium may store one or more programs including instructions that, when individually or collectively executed by the at least one processor (501), cause the wearable device (400) to receive, from the external electronic device, information for displaying a screen associated with the external electronic device.The non-transitory computer-readable storage medium may store one or more programs including instructions that, when individually or collectively executed by the at least one processor (501), cause the wearable device (400) to display, through the display (503), a visual object generated based on the information and at least partially superimposed on the target object.

[0211] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0212] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0213] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0214] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0215] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0216] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In a wearable device (400), A memory (507) storing instructions and including one or more storage media; display (503); communication circuit (505); and At least one processor (501) comprising a processing circuit; The above instructions, when individually or collectively executed by the at least one processor (501), cause the wearable device (400) to: Connecting to one or more external electronic devices through the above communication circuit (505); Based on identifying a specified event, a target object is identified among one or more external objects within the field of view (FoV) of the wearable device (400); Identifying an external electronic device among the one or more external electronic devices based on an attribute of the target object; Receiving information for displaying a screen associated with the external electronic device from the external electronic device; and Causing a visual object generated based on the above information and at least partially superimposed on the target object to be displayed through the display (503). Wearable device (400).

2. In claim 1, One or more of the above external electronic devices: An electronic device connected to the above wearable device (400) via short-range communication, An electronic device containing the same account information as the wearable device (400), or At least one server connected to another external electronic device and managing said other external electronic device; Wearable device (400).

3. In claim 1, The above specified events are: A first time period in which the user's gaze of the wearable device (400) is positioned within a specified range with respect to the target object, which is greater than or equal to the first reference time period; A second time during which the user's body part is located within the specified range, which is longer than the second reference time, or comprising at least one of the specified gestures for the specified range, Wearable device (400).

4. In claim 1, wherein said one or more external electronic devices are one or more first external electronic devices, The above instructions, when individually or collectively executed by the at least one processor (501), cause the wearable device (400) to: Identifying one or more second external electronic devices through the above communication circuit (505); Identifying a third external electronic device within the FoV among the one or more second external electronic devices; and causing said target object to be identified among said one or more external objects including said third external electronic device; Wearable device (400).

5. In claim 4, The one or more second external electronic devices: An external electronic device connected to an AP (access point) to which the above wearable device (400) is connected, An external electronic device whose signal strength received by the wearable device (400) is greater than or equal to the reference strength, or At least one of the external electronic devices, the distance from the wearable device (400) identified based on the positioning technique being less than a reference distance, Wearable device (400).

6. In claim 4, The above instructions, when individually or collectively executed by the at least one processor (501), cause the wearable device (400) to: Receiving a signal from a server connected to said one or more second external electronic devices, wherein said signal instructs said one or more second external electronic devices; Based on said signal, causing said one or more second external electronic devices to be identified, The signal is transmitted from the server in response to another signal indicating that the one or more second external electronic devices identify the wearable device (400). Wearable device (400).

7. In claim 4, The external electronic device identified among the one or more external electronic devices is the third external electronic device, The above instructions, when individually or collectively executed by the at least one processor (501), cause the wearable device (400) to: Identifying the third external electronic device as the target object among the one or more external objects including the third external electronic device; Receiving said information from said third external electronic device; and Causing the visual object to be displayed through the display (503) while another display of the third external electronic device is turned off. Wearable device (400).

8. In claim 7, The above instructions, when individually or collectively executed by the at least one processor (501), cause the wearable device (400) to: Receive information for displaying another screen to be changed from the above screen, from the third external electronic device; ceasing display of said visual object; and Causing another visual object, generated based on information for displaying the other screen and at least partially overlapping with the target object, to be displayed through the display (503), Information for displaying the other screen is generated based on touch input to the other display while the other display is turned off. Wearable device (400).

9. In claim 1, The above property includes at least one of the size of the target object, the material of the target object, or the type of the target object, If the above property is the above size, the external electronic device represents an external electronic device among the one or more external electronic devices that has the smallest difference between the size of the display area and the size of the target object. Wearable device (400).

10. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (501), cause the wearable device (400) to: Based on identifying another event specified, transmitting a request signal for said information to said external electronic device; Causing to receive, from the external electronic device, a response signal to the request signal including the above information; Wearable device (400).

11. In claim 1, wherein said one or more external electronic devices include a plurality of external electronic devices, The above plurality of external electronic devices include the above external electronic device and another external electronic device, The above instructions, when individually or collectively executed by the at least one processor (501), cause the wearable device (400) to: Displaying the visual object on the first partial area of ​​the target object; and Causing to display a different visual object on a second sub-area different from the first sub-area of ​​the target object, The above other visual object corresponds to another screen to be displayed on the other external electronic device. Wearable device (400).

12. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (501), cause the wearable device (400) to: Obtaining input for some area of ​​the above visual object; Transmitting information about said certain areas to said external electronic device; Receiving, from the external electronic device, information for displaying another screen to be changed from the screen based on information about the above-mentioned partial area; ceasing display of said visual object; and Causing another visual object, generated based on information for displaying the other screen and at least partially overlapping with the target object, to be displayed through the display (503). Wearable device (400).

13. In claim 1, The one or more external electronic devices include a plurality of external electronic devices, The above instructions, when individually or collectively executed by the at least one processor (501), cause the wearable device (400) to: If the above property is the size of the target object, identify that the size of the target object is changed from the first size to the second size; Based on the second size, identifying another external electronic device corresponding to the second size among the plurality of external electronic devices; Receive information for displaying another screen to be displayed on the other external electronic device from the other external electronic device; ceasing display of said visual object; and Causing another visual object, generated based on information for displaying the other screen and at least partially overlapping with the target object, to be displayed through the display (503). Wearable device (400).

14. A method performed by a wearable device (400), wherein the method comprises: The act of connecting with one or more external electronic devices; An operation of identifying a target object among one or more external objects within the field of view (FoV) of the wearable device (400) based on identifying a specified event; An action of identifying an external electronic device among the one or more external electronic devices based on an attribute of the target object; An operation of receiving information for displaying a screen associated with the external electronic device from the external electronic device; and comprising an action of displaying a visual object generated based on said information and at least partially superimposed on said target object; method.

15. A non-transitory computer-readable storage medium, when individually or collectively executed by at least one processor (501) of a wearable device (400) including a display (503) and a communication circuit (505), causes the wearable device (400) to: Connecting to one or more external electronic devices through the above communication circuit (505); Based on identifying a specified event, a target object is identified among one or more external objects within the field of view (FoV) of the wearable device (400); Identifying an external electronic device among the one or more external electronic devices based on an attribute of the target object; Receiving information for displaying a screen associated with the external electronic device from the external electronic device; and storing one or more programs including instructions that cause a visual object generated based on said information and at least partially superimposed on said target object to be displayed through said display (503); A non-transitory computer-readable storage medium.

Images