Electronic device and method for displaying connection state in virtual environment
The wearable device's ability to visually represent and manage connection states between wearable and external electronic devices enhances user experience and usability in XR environments by providing clear visual feedback and intuitive interaction with connection states.
Patent Information
- Application Number
- PCT/KR2024/004846
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-08-24
- Filing Date
- 2024-04-11
- Publication Date
- 2025-01-23
AI Technical Summary
Current electronic devices lack an effective method to visually indicate and manage connection status between wearable devices and external electronic devices in virtual environments, leading to user confusion and decreased usability in extended reality (XR) experiences.
A wearable device equipped with a display, camera, and communication circuitry identifies external electronic devices, establishes connections, and displays visual objects representing connection states, allowing users to interact with these states through input, thereby enhancing user experience and usability in XR environments.
The solution provides clear visual feedback on connection states, improving user interaction and immersion in XR environments by enabling intuitive management of connections between wearable devices and external electronic devices.
Smart Images

Figure KR2024004846_23012025_PF_FP_ABST
Abstract
Description
Electronic device and method for indicating connection status in a virtual environment
[0001] The present disclosure relates to an electronic device and method for indicating a connection state in a virtual environment.
[0002] In order to provide an enhanced user experience, electronic devices are being developed that provide extended reality (XR) (e.g., augmented reality (AR), virtual reality (VR)) services that display computer-generated information in conjunction with external objects in the real world or virtual objects in the virtual world. The electronic devices may include wearable devices that can be worn by a user. For example, the electronic devices may include user equipment, AR glasses, VR glasses, or a head-mounted device (HMD) (e.g., video see through (VST) HMD, optical see through (OST) HMD).
[0003] According to one aspect of the present disclosure, a wearable device may include a display. The wearable device may include communication circuitry. The wearable device may include a camera. The wearable device may include at least one processor including a processing circuit. The wearable device may include one or more storage media and a memory storing instructions. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify a plurality of external electronic devices using the communication circuitry. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to establish a first connection with a first external electronic device among the plurality of external electronic devices using the communication circuitry. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to acquire an image of the surroundings of the wearable device through the camera. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify a second external electronic device in the image from among the plurality of external electronic devices. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display, through the display, a first visual object representing the first connection based on identifying an event associated with the first connection.The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to obtain an input for the first visual object for establishing a second connection between the first external electronic device and the second external electronic device. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to, based on the input, cease displaying the first visual object and display, through the display, a second visual object representing the second connection.
[0004] According to one aspect of the present disclosure, a method performed by a wearable device may include an operation of identifying a plurality of external electronic devices. The method may include an operation of establishing a first connection with a first external electronic device among the plurality of external electronic devices. The method may include an operation of acquiring an image of an environment of the wearable device. The method may include an operation of identifying a second external electronic device in the acquired image among the plurality of external electronic devices. The method may include an operation of displaying a first visual object representing the first connection based on identifying an event associated with the first connection. The method may include an operation of acquiring an input for the first visual object for establishing a second connection between the first external electronic device and the second external electronic device. The method may include an operation of ceasing display of the first visual object and displaying a second visual object representing the second connection based on the input.
[0005] According to one aspect of the present disclosure, a non-transitory computer-readable storage medium may store one or more programs comprising instructions that, when individually or collectively executed by at least one processor of a wearable device including a display, a camera, and communication circuitry, cause the device to identify a plurality of external electronic devices using the communication circuitry. The non-transitory computer-readable storage medium may store one or more programs comprising instructions that, when individually or collectively executed by the at least one processor, cause the device to establish a first connection with a first external electronic device among the plurality of external electronic devices using the communication circuitry. The non-transitory computer-readable storage medium may store one or more programs comprising instructions that, when individually or collectively executed by the at least one processor, cause the device to acquire an image of an environment of the wearable device through the camera. The non-transitory computer-readable storage medium may store one or more programs comprising instructions that, when individually or collectively executed by the at least one processor, cause the at least one processor to identify a second external electronic device in the acquired image from among the plurality of external electronic devices. The non-transitory computer-readable storage medium may store one or more programs comprising instructions that, when individually or collectively executed by the at least one processor, cause the at least one processor to display, through the display, a first visual object representing the first connection based on identifying an event associated with the first connection.The non-transitory computer-readable storage medium may store one or more programs comprising instructions that, when individually or collectively executed by the at least one processor, cause the at least one processor to obtain an input for the first visual object for establishing a second connection between the first external electronic device and the second external electronic device. The non-transitory computer-readable storage medium may store one or more programs comprising instructions that, when individually or collectively executed by the at least one processor, cause the at least one processor to, based on the input, cease displaying the first visual object and display, through the display, a second visual object representing the second connection.
[0006] The above and other aspects, features, and advantages of specific embodiments of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:
[0007] FIG. 1 is a block diagram of an electronic device within a network environment according to one or more embodiments;
[0008] FIG. 2A illustrates an example of a perspective view of a wearable device according to one or more embodiments;
[0009] FIG. 2b illustrates an example of one or more hardware elements arranged within a wearable device according to one or more embodiments;
[0010] FIGS. 3A and 3B illustrate examples of the appearance of a wearable device according to one or more embodiments;
[0011] FIG. 4A illustrates an example of a method for displaying a visual object indicating a connection status between a wearable device and an external electronic device;
[0012] FIG. 4b illustrates an example of a method for displaying a visual object indicating a connection status between external electronic devices;
[0013] FIG. 5 illustrates an exemplary block diagram of a wearable device within a network environment;
[0014] FIGS. 6A and 6B illustrate examples of a method for displaying a visual object indicating the connection status of an external electronic device;
[0015] FIG. 7 illustrates an example of how a wearable device identifies multiple external electronic devices;
[0016] FIGS. 8A and 8B illustrate examples of a method for displaying visual objects indicating the connection status of an external electronic device supporting multiple connections;
[0017] FIG. 9 illustrates an example of a method for establishing a connection based on an input for a visual object indicating a connection status and displaying a visual object according to the established connection;
[0018] FIG. 10A illustrates an example of a method for displaying a visual object indicating a connection status of an external electronic device and displaying another visual object based on an input to the visual object;
[0019] FIG. 10b illustrates an example of a method for displaying a visual object indicating the connection status of an external electronic device and disconnecting the connection based on input to the visual object;
[0020] Figure 11 illustrates examples of visual objects representing connection status;
[0021] FIG. 12 illustrates an example of a method for displaying a visual object indicating a connection status between a wearable device and an external electronic device; and
[0022] FIG. 13 illustrates an example of a method for displaying a visual object indicating a connection status between external electronic devices.
[0023] The terms used in this disclosure are used only to describe specific embodiments and may not be intended to limit the scope of other embodiments. The singular expression may include plural expressions unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as commonly understood by those of ordinary skill in the art described in this disclosure. Terms defined in general dictionaries among the terms used in this disclosure may be interpreted as having the same or similar meaning in the context of the relevant technology, and shall not be interpreted in an idealized or overly formal sense unless explicitly defined in this disclosure. In some cases, even if a term is defined in this disclosure, it cannot be interpreted to exclude embodiments of the present disclosure.
[0024] One or more embodiments of the present disclosure described below illustrate a hardware-based approach. However, since one or more embodiments of the present disclosure include techniques utilizing both hardware and software, various embodiments of the present disclosure do not exclude software-based approaches.
[0025] In addition, in the present disclosure, expressions such as "more than" or "less than" may be used to determine whether a specific condition is satisfied or fulfilled. However, this is merely a description for expressing an example and does not exclude descriptions such as "more than" or "less than." Conditions described as "more than" may be replaced with "more than," conditions described as "less than," and conditions described as "more than and less than" may be replaced with "more than and less than." In addition, hereinafter, "A" to "B" mean at least one of the elements from A (including A) to B (including B).
[0026] The term "couple" and its derivatives mean direct or indirect communication between two or more elements, whether or not the two or more elements are in physical contact with each other. The terms "transmit," "receive," and "communicate," and their derivatives, include both direct and indirect communication. The terms "include" and "comprise," and their derivatives, mean including without limitation. The term "or" is an inclusive term meaning "and / or." The phrase "associated with" and its derivatives refer to include, be included within, interconnect with, contain, be contained within, connect to or with, couple to or with, be communicable with, cooperate with, interleave, juxtapose, be proximate to, be bound to or with, have, have a property of, have a relationship to or with, etc.
[0027] The phrase "at least one of" used with a list of items means that various combinations of one or more of the listed items may be used, and only one item from the list may be required. For example, "at least one of A, B, C" includes combinations of A, B, C, A and B, A and C, B and C, and A, B, and C, as well as all variations thereof. Similarly, the term "set" means one or more. Thus, a set of items may be a single item or a collection of two or more items.
[0028] FIG. 1 is a block diagram of an electronic device within a network environment, according to one or more embodiments.
[0029] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).
[0030] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or calculations. According to one embodiment, as at least a part of the data processing or calculations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or a secondary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor)) that can operate independently or together therewith. For example, if the electronic device (101) includes a main processor (121) and a secondary processor (123), the secondary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a specified function. The secondary processor (123) may be implemented separately from the main processor (121) or as a part thereof.
[0031] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.
[0032] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).
[0033] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).
[0034] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).
[0035] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.
[0036] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. In one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.
[0037] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).
[0038] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.
[0039] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.
[0040] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).
[0041] A haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. In one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.
[0042] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.
[0043] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least a part of a power management integrated circuit (PMIC).
[0044] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.
[0045] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).
[0046] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.
[0047] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas by, for example, the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).
[0048] According to one or more embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band.
[0049] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).
[0050] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.
[0051] FIG. 2A illustrates an example of a perspective view of a wearable device according to one or more embodiments. FIG. 2B illustrates an example of one or more hardware components arranged within a wearable device according to one or more embodiments.
[0052] According to one embodiment, the wearable device (103) may have the form of glasses that are wearable on a body part of the user (e.g., the head). The wearable device (103) of FIGS. 2A and 2B may be an example of the electronic device (101) of FIG. 1. The wearable device (103) may include a head-mounted display (HMD). For example, the housing of the wearable device (103) may include a flexible material, such as rubber and / or silicone, that is configured to fit closely to a portion of the user's head (e.g., a portion of the face surrounding both eyes). For example, the housing of the wearable device (103) may include one or more straps that are capable of being twined around the user's head, and / or one or more temples that are detachably attachable to the ears of the head.
[0053] Referring to FIG. 2A, according to one embodiment, a wearable device (103) may include at least one display (250) and a frame (200) supporting at least one display (250).
[0054] According to one embodiment, a wearable device (103) can be worn on a part of a user's body. The wearable device (103) can provide extended reality (XR) to a user wearing the wearable device (103). For example, the extended reality can provide augmented reality (AR), virtual reality (VR), or mixed reality (MR) that combines augmented reality and virtual reality. For example, the wearable device (103) can display a virtual reality image provided from at least one optical device (282, 284) of FIG. 2B on at least one display (250) in response to a user's designated gesture acquired through the motion recognition cameras (260-2, 263) of FIG. 2B.
[0055] According to one embodiment, at least one display (250) may provide visual information to a user. For example, at least one display (250) may include a transparent or translucent lens. At least one display (250) may include a first display (250-1) and / or a second display (250-2) spaced apart from the first display (250-1). For example, the first display (250-1) and the second display (250-2) may be positioned at positions corresponding to the user's left and right eyes, respectively.
[0056] Referring to FIG. 2B, at least one display (250) can provide visual information transmitted from external light to a user through a lens included in at least one display (250), and other visual information distinct from the visual information. The lens can be formed based on at least one of a Fresnel lens, a pancake lens, or a multi-channel lens. For example, at least one display (250) can include a first surface (231) and a second surface (232) opposite to the first surface (231). A display area can be formed on the second surface (232) of at least one display (250). When a user wears the wearable device (103), external light can be transmitted to the user by being incident on the first surface (231) and transmitted through the second surface (232). As another example, at least one display (250) can display an augmented reality image combined with a virtual reality image provided from at least one optical device (282, 284) on a real screen transmitted through external light, in a display area formed on the second surface (232).
[0057] In one embodiment, at least one display (250) may include at least one waveguide (233, 234) that diffracts light emitted from at least one optical device (282, 284) and transmits the diffracted light to a user. The at least one waveguide (233, 234) may be formed based on at least one of glass, plastic, or polymer. A nano-pattern may be formed on at least a portion of the exterior or interior of the at least one waveguide (233, 234). The nano-pattern may be formed based on a grating structure having a polygonal and / or curved shape. Light incident on one end of the at least one waveguide (233, 234) may be propagated to the other end of the at least one waveguide (233, 234) by the nano-pattern. At least one waveguide (233, 234) may include at least one diffractive element (e.g., a diffractive optical element (DOE), a holographic optical element (HOE)) and at least one reflective element (e.g., a reflective mirror). For example, at least one waveguide (233, 234) may be arranged within the wearable device (103) to guide a screen displayed by at least one display (250) to the user's eyes. For example, the screen may be transmitted to the user's eyes based on total internal reflection (TIR) occurring within the at least one waveguide (233, 234).
[0058] The wearable device (103) can analyze an object included in a real image collected through a shooting camera (260-4), combine a virtual object corresponding to an object to be provided with augmented reality among the analyzed objects, and display the virtual object on at least one display (250). The virtual object can include at least one of text and an image regarding various information related to the object included in the real image. The wearable device (103) can analyze the object based on a multi-camera such as a stereo camera. For the object analysis, the wearable device (103) can perform spatial recognition (e.g., simultaneous localization and mapping (SLAM)) using a multi-camera and / or time-of-flight (ToF). A user wearing the wearable device (103) can view an image displayed on at least one display (250).
[0059] According to one embodiment, the frame (200) may be configured as a physical structure that allows the wearable device (103) to be worn on the user's body. According to one embodiment, the frame (200) may be configured so that, when the user wears the wearable device (103), the first display (250-1) and the second display (250-2) can be positioned corresponding to the user's left and right eyes. The frame (200) may support at least one display (250). For example, the frame (200) may support the first display (250-1) and the second display (250-2) to be positioned corresponding to the user's left and right eyes.
[0060] Referring to FIG. 2A, the frame (200) may include a region (220) that is in contact with at least a portion of a user's body when the user wears the wearable device (103). For example, the region (220) of the frame (200) that is in contact with a portion of the user's body may include a region that is in contact with a portion of the user's nose, a portion of the user's ear, and a portion of the side of the user's face that the wearable device (103) makes contact with. According to one embodiment, the frame (200) may include a nose pad (210) that is in contact with a portion of the user's body. When the wearable device (103) is worn by the user, the nose pad (210) may be in contact with a portion of the user's nose. The frame (200) may include a first temple (204) and a second temple (205) that are in contact with another portion of the user's body that is distinct from the portion of the user's body.
[0061] For example, the frame (200) may include a first rim (201) that surrounds at least a portion of the first display (250-1), a second rim (202) that surrounds at least a portion of the second display (250-2), a bridge (203) that is disposed between the first rim (201) and the second rim (202), a first pad (211) that is disposed along a portion of the edge of the first rim (201) from one end of the bridge (203), a second pad (212) that is disposed along a portion of the edge of the second rim (202) from the other end of the bridge (203), a first temple (204) that extends from the first rim (201) and is fixed to a portion of the wearer's ear, and a second temple (205) that extends from the second rim (202) and is fixed to a portion of the ear opposite the ear. The first pad (211) and the second pad (212) may be in contact with a portion of the user's nose, and the first temple (204) and the second temple (205) may be in contact with a portion of the user's face and a portion of the user's ear. The temples (204, 205) may be rotatably connected to the rim through the hinge units (206, 207) of FIG. 2B. The first temple (204) may be rotatably connected to the first rim (201) through the first hinge unit (206) disposed between the first rim (201) and the first temple (204). The second temple (205) may be rotatably connected to the second rim (202) through the second hinge unit (207) disposed between the second rim (202) and the second temple (205). According to one embodiment, the wearable device (103) can identify an external object (e.g., a user's fingertip) touching the frame (200) and / or a gesture performed by the external object by using a touch sensor, a grip sensor, and / or a proximity sensor formed on at least a portion of a surface of the frame (200).
[0062] According to one embodiment, the wearable device (103) may include hardwares (e.g., hardwares to be described later in FIG. 5) that perform various functions. For example, the hardwares may include a battery module (270), an antenna module (275), at least one optical device (282, 284), speakers (e.g., speakers 255-1, 255-2), a microphone (e.g., microphones 265-1, 265-2, 265-3), a light-emitting module (not shown), and / or a printed circuit board (PCB) (290) (e.g., a printed circuit board). The various hardwares may be arranged within the frame (200).
[0063] According to one embodiment, the microphones (e.g., microphones 265-1, 265-2, 265-3) of the wearable device (103) may be disposed on at least a portion of the frame (200) to acquire sound signals. A first microphone (265-1) disposed on the bridge (203), a second microphone (265-2) disposed on the second rim (202), and a third microphone (265-3) disposed on the first rim (201) are illustrated in FIG. 2B , but the number and arrangement of the microphones (265) are not limited to the embodiment of FIG. 2B . When the number of microphones (265) included in the wearable device (103) is two or more, the wearable device (103) may identify the direction of the sound signal by using a plurality of microphones disposed on different portions of the frame (200).
[0064] According to one embodiment, at least one optical device (282, 284) can project a virtual object onto at least one display (250) to provide various image information to a user. For example, at least one optical device (282, 284) can be a projector. At least one optical device (282, 284) can be disposed adjacent to at least one display (250) or can be included within at least one display (250) as a part of at least one display (250). According to one embodiment, the wearable device (103) can include a first optical device (282) corresponding to a first display (250-1) and a second optical device (284) corresponding to a second display (250-2). For example, at least one optical device (282, 284) may include a first optical device (282) disposed at an edge of a first display (250-1) and a second optical device (284) disposed at an edge of a second display (250-2). The first optical device (282) may transmit light to a first waveguide (233) disposed on the first display (250-1), and the second optical device (284) may transmit light to a second waveguide (234) disposed on the second display (250-2).
[0065] In one embodiment, the camera (260) may include a recording camera (260-4), an eye tracking camera (ET CAM) (260-1), and / or a motion recognition camera (260-2, 260-3). The recording camera (260-4), the eye tracking camera (260-1), and the motion recognition cameras (260-2, 260-3) may be positioned at different locations on the frame (200) and may perform different functions. The eye tracking camera (260-1) may output data indicating the position or gaze of the eyes of a user wearing the wearable device (103). For example, the wearable device (103) may detect the gaze from an image including the user's pupils obtained through the eye tracking camera (260-1). The wearable device (103) can perform gaze interaction with at least one object using the user's gaze acquired through the gaze tracking camera (260-1). The wearable device (103) can express a portion corresponding to the eye of an avatar representing the user in a virtual space using the user's gaze acquired through the gaze tracking camera (260-1). The wearable device (103) can render an image (or screen) displayed on at least one display (250) based on the position of the user's eyes. For example, the visual quality (e.g., resolution, brightness, saturation, grayscale, PPI) of a first region related to the gaze and a second region distinguished from the first region within the image may be different from each other. For example, if the wearable device (103) supports an iris recognition function, user authentication can be performed based on iris information acquired using the gaze tracking camera (260-1).An example in which the gaze tracking camera (260-1) is positioned toward the user's right eye is illustrated in FIG. 2B, but the embodiment is not limited thereto, and the gaze tracking camera (260-1) may be positioned solely toward the user's left eye, or may be positioned toward both eyes.
[0066] In one embodiment, the capturing camera (260-4) can capture an actual image or background to be aligned with a virtual image in order to implement augmented reality or mixed reality content. The capturing camera (260-4) can capture an image of a specific object existing at a location viewed by the user and provide the image to at least one display (250). The at least one display (250) can display a single image in which information about an actual image or background including an image of the specific object acquired using the capturing camera (260-4) is superimposed on a virtual image provided through at least one optical device (282, 284). The wearable device (103) can compensate for depth information (e.g., the distance between the wearable device (103) and an external object acquired using a depth sensor) using the image acquired using the capturing camera (260-4). The wearable device (103) can perform object recognition using the image acquired using the capturing camera (260-4). The wearable device (103) may perform a pass-through function to display an image acquired through a photographing camera (260-4) by overlaying it on at least a portion of a screen representing a virtual space on at least one display (250). In one embodiment, the photographing camera (260-4) may be positioned on a bridge (203) positioned between a first rim (201) and a second rim (202).
[0067] The gaze tracking camera (260-1) can implement more realistic augmented reality by tracking the gaze of a user wearing the wearable device (103) and thereby matching the user's gaze with visual information provided to at least one display (250). For example, when the wearable device (103) looks straight ahead, the wearable device (103) can naturally display environmental information related to the user's front at a location where the user is located on at least one display (250). The gaze tracking camera (260-1) can be configured to capture an image of the user's pupil to determine the user's gaze. For example, the gaze tracking camera (260-1) can receive gaze detection light reflected from the user's pupil and track the user's gaze based on the position and movement of the received gaze detection light. In one embodiment, the gaze tracking camera (260-1) can be positioned at positions corresponding to the user's left and right eyes. For example, the gaze tracking camera (260-1) may be positioned within the first rim (201) and / or the second rim (202) to face the direction in which the user wearing the wearable device (103) is positioned.
[0068] The motion recognition camera (260-2, 260-3) can recognize the movement of the user's entire body, such as the user's torso, hand, or face, or a part of the body, and thereby provide a specific event on a screen provided on at least one display (250). The motion recognition camera (260-2, 260-3) can recognize the user's motion (gesture recognition), obtain a signal corresponding to the motion, and provide a display corresponding to the signal on at least one display (250). The wearable device (103) can identify the signal corresponding to the motion, and perform a designated function based on the identification. The motion recognition camera (260-2, 260-3) can be used to perform a spatial recognition function using SLAM and / or a depth map for 6 degrees of freedom pose (6 dof pose). The wearable device (103) can perform gesture recognition and / or object tracking functions using motion recognition cameras (260-2, 260-3). In one embodiment, the motion recognition cameras (260-2, 260-3) can be positioned on the first rim (201) and / or the second rim (202).
[0069] The camera (260) included in the wearable device (103) is not limited to the above-described gaze tracking camera (260-1) and motion recognition cameras (260-2, 260-3). For example, the wearable device (103) can identify an external object included in the FoV using a camera positioned toward the user's FoV. The wearable device (103) can identify an external object based on a sensor for identifying the distance between the wearable device (103) and the external object, such as a depth sensor and / or a time of flight (ToF) sensor. The camera (260) positioned toward the FoV can support an autofocus function and / or an optical image stabilization (OIS) function. For example, the wearable device (103) may include a camera (260) (e.g., a face tracking (FT) camera) positioned toward the face to obtain an image including the face of a user wearing the wearable device (103).
[0070] In one embodiment, the wearable device (103) may further include a light source (e.g., an LED) that emits light toward a subject (e.g., a user's eyes, face, and / or an external object within the FoV) being captured using the camera (260). The light source may include an infrared wavelength LED. The light source may be disposed on at least one of the frame (200) and the hinge units (206, 207).
[0071] According to one embodiment, the battery module (270) may supply power to electronic components of the wearable device (103). In one embodiment, the battery module (270) may be disposed within the first temple (204) and / or the second temple (205). For example, the battery module (270) may be a plurality of battery modules (270). The plurality of battery modules (270) may be disposed within each of the first temple (204) and the second temple (205). In one embodiment, the battery module (270) may be disposed at an end of the first temple (204) and / or the second temple (205).
[0072] The antenna module (275) can transmit signals or power to the outside of the wearable device (103), or receive signals or power from the outside. In one embodiment, the antenna module (275) can be positioned within the first temple (204) and / or the second temple (205). For example, the antenna module (275) can be positioned close to one surface of the first temple (204) and / or the second temple (205).
[0073] The speaker (255) can output an audio signal to the outside of the wearable device (103). The audio output module may be referred to as a speaker. In one embodiment, the speaker (255) may be positioned within the first temple (204) and / or the second temple (205) so as to be positioned adjacent to the ear of a user wearing the wearable device (103). For example, the speaker (255) may include a second speaker (255-2) positioned within the first temple (204) and thus positioned adjacent to the user's left ear, and a first speaker (255-1) positioned within the second temple (205) and thus positioned adjacent to the user's right ear.
[0074] The light-emitting module may include at least one light-emitting element. The light-emitting module may emit light of a color corresponding to a specific state or emit light with an action corresponding to a specific state to visually provide information regarding a specific state of the wearable device (103) to the user. For example, when the wearable device (103) requires charging, the wearable device (103) may emit red light at a regular cycle. In one embodiment, the light-emitting module may be disposed on the first rim (201) and / or the second rim (202).
[0075] Referring to FIG. 2B, according to one embodiment, a wearable device (103) may include a printed circuit board (PCB) (290). The PCB (290) may be included in at least one of the first temple (204) or the second temple (205). The PCB (290) may include an interposer disposed between at least two sub-PCBs. One or more hardwares included in the wearable device (103) (e.g., hardwares to be described later in FIG. 5) may be disposed on the PCB (290). The wearable device (103) may include a flexible PCB (FPCB) for interconnecting the hardwares.
[0076] According to one embodiment, the wearable device (103) may include at least one of a gyro sensor, a gravity sensor, and / or an acceleration sensor for detecting a posture of the wearable device (103) and / or a posture of a body part (e.g., a head) of a user wearing the wearable device (103). Each of the gravity sensor and the acceleration sensor may measure gravitational acceleration and / or acceleration based on mutually perpendicular designated three-dimensional axes (e.g., an x-axis, a y-axis, and a z-axis). The gyro sensor may measure an angular velocity of each of the designated three-dimensional axes (e.g., an x-axis, a y-axis, and a z-axis). At least one of the gravity sensor, the acceleration sensor, and the gyro sensor may be referred to as an inertial measurement unit (IMU). According to one embodiment, the wearable device (103) may identify a user's motion and / or gesture performed to execute or cease a specific function of the wearable device (103) based on the IMU.
[0077] FIGS. 3A and 3B illustrate examples of the appearance of a wearable device according to one or more embodiments.
[0078] The wearable device (103) of FIGS. 3A and 3B may be an example of the electronic device (101) of FIG. 1. According to one embodiment, an example of the appearance of a first side (310) of a housing of the wearable device (103) may be illustrated in FIG. 3A, and an example of the appearance of a second side (320) opposite to the first side (310) may be illustrated in FIG. 3B.
[0079] Referring to FIG. 3A, a first surface (310) of a wearable device (103) according to one embodiment may have a form attachable to a body part of a user (e.g., the face of the user). The wearable device (103) may further include a strap for fixing to a body part of the user, and / or one or more temples (e.g., the first temple (204) and / or the second temple (205) of FIGS. 2A and 2B). A first display (250-1) for outputting an image to a left eye among the user's two eyes, and a second display (250-2) for outputting an image to a right eye among the user's two eyes may be disposed on the first surface (310). The wearable device (103) is formed on the first surface (310) and may further include a rubber or silicone packing to prevent interference by light (e.g., ambient light) different from the light emitted from the first display (250-1) and the second display (250-2).
[0080] According to one embodiment, the wearable device (103) may include cameras (260-1) for photographing and / or tracking both eyes of the user adjacent to each of the first display (250-1) and the second display (250-2). The cameras (260-1) may be referred to as the gaze tracking camera (260-1) of FIG. 2B. According to one embodiment, the wearable device (103) may include cameras (260-5, 260-6) for photographing and / or recognizing the face of the user. The cameras (260-5, 260-6) may be referred to as FT cameras. The wearable device (103) may control an avatar representing the user in a virtual space based on the motion of the user's face identified using the cameras (260-5, 260-6).
[0081] Referring to FIG. 3b, a camera (e.g., cameras (260-7, 260-8, 260-9, 260-10, 260-11, 260-12)) and / or a sensor (e.g., a depth sensor (330)) for obtaining information related to the external environment of the wearable device (103) may be disposed on a second surface (320) opposite to the first surface (310) of FIG. 3a. For example, the cameras (260-7, 260-8, 260-9, 260-10) may be disposed on the second surface (320) to recognize external objects. Cameras (260-7, 260-8, 260-9, 260-10) may be referenced to the motion recognition cameras (260-2, 260-3) of FIG. 2B.
[0082] For example, using cameras (260-11, 260-12), the wearable device (103) can obtain images and / or videos to be transmitted to each of the user's eyes. The camera (260-11) can be placed on the second face (320) of the wearable device (103) to obtain an image to be displayed through the second display (250-2) corresponding to the right eye among the two eyes. The camera (260-12) can be placed on the second face (320) of the wearable device (103) to obtain an image to be displayed through the first display (250-1) corresponding to the left eye among the two eyes. The cameras (260-11, 260-12) can be referred to as the shooting camera (260-4) of FIG. 2B.
[0083] According to one embodiment, the wearable device (103) may include a depth sensor (330) disposed on the second face (320) to identify a distance between the wearable device (103) and an external object. Using the depth sensor (330), the wearable device (103) may obtain spatial information (e.g., a depth map) for at least a portion of the FoV of a user wearing the wearable device (103). A microphone may be disposed on the second face (320) of the wearable device (103) to obtain sound output from an external object. The number of microphones may be one or more, depending on the embodiment.
[0084] An electronic device can establish a connection with one or more external electronic devices based on a communication technique. For example, the communication technique may include short-range communication. The short-range communication may be utilized. For example, the short-range communication may include BT (Bluetooth), BLE (Bluetooth low energy), Wi-Fi (Wireless fidelity), or UWB (ultra-wideband). A connection between electronic devices using the short-range communication may be established based on manipulation (or control) of the electronic device. For example, for a BT connection between a smartphone and a headset, the connection may be performed by activating the BT function of both devices and selecting the device identified based on the scan results. Furthermore, if the device to be connected is already connected to another device, there may be the inconvenience of terminating the connection between the existing connected devices and performing an operation for a new connection.
[0085] An electronic device and method according to embodiments of the present disclosure can visually provide a user of the wearable device (103) with a connection status (or connection information) by displaying a visual object (e.g., an image or animation) regarding a connection status (or connection information) between an external electronic device and the wearable device (103) or between external electronic devices in a wearable device (103) that provides extended reality. In addition, the electronic device and method according to embodiments of the present disclosure can cause a change in the connection status based on obtaining a user's input regarding the visual object. Accordingly, the electronic device and method according to embodiments of the present disclosure can improve usability in an XR environment and provide an immersive user experience to the user by processing the user's input.
[0086] FIG. 4A illustrates an example of a method for displaying a visual object indicating a connection status between a wearable device and an external electronic device.
[0087] The wearable device (103) of FIG. 4A may represent an example of the electronic device (101) of FIG. 1. FIG. 4A illustrates an embodiment (400) in which a wearable device (103) is connected to a plurality of external electronic devices (421, 422, 423) (TV, tablet, smartphone).
[0088] Referring to embodiment (400), the wearable device (103) can identify a plurality of external electronic devices (421, 422, 423). For example, the wearable device (103) can identify a plurality of external electronic devices (421, 422, 423) within a field of view (FoV) (407) of the wearable device (103). However, the embodiment of the present disclosure is not limited thereto. The FoV (407) of the wearable device (103) can include a real environment (e.g., AR environment) seen through a camera of the wearable device (103) or an image rendered for the real environment (e.g., VR environment). For example, the wearable device (103) can identify a plurality of external electronic devices based on a communication circuit (e.g., a communication module (190) of FIG. 1) included in the wearable device (103). The plurality of external electronic devices may include a plurality of external electronic devices (421, 422, 423). For example, the wearable device (103) can identify the plurality of external electronic devices based on the communication circuit, and identify the plurality of external electronic devices (421, 422, 423) within the FoV (407) among the plurality of external electronic devices.
[0089] For example, based on the communication circuit, identifying the plurality of external electronic devices may utilize short-range communication. For example, the short-range communication may include BT (Bluetooth), BLE (Bluetooth low energy), Wi-Fi (Wireless fidelity), or UWB (ultra-wideband). However, the embodiments of the present disclosure are not limited thereto. For example, the wearable device (103) may identify the plurality of external electronic devices based on a wireless communication or wired communication technology that is different from the short-range communication. Identifying the plurality of external electronic devices may be referred to as searching, discovering, detecting, or scanning the plurality of external electronic devices.
[0090] Referring to embodiment (400), the wearable device (103) can establish a connection with at least one external electronic device among a plurality of external electronic devices (421, 422, 423). For example, the wearable device (103) can establish a connection with the external electronic device (421) and the external electronic device (422). For example, the wearable device (103) may not establish a connection with the external electronic device (423). In other words, even if the wearable device (103) identifies the external electronic device (423), it may not establish a connection with the identified external electronic device (423). In the above-described embodiment, some external electronic devices are illustrated as being connected to the wearable device (103), but the embodiments of the present disclosure are not limited thereto.
[0091] According to one embodiment, the wearable device (103) may display a visual object indicating a connection status. For example, the wearable device (103) may display a visual object (431) indicating a connection established with an external electronic device (421). For example, the wearable device (103) may display the visual object (431) indicating the connection between the wearable device (103) and the external electronic device (423) through a display (e.g., the display module (160) of FIG. 1). For example, if the screen displayed through the display is from a third-person viewpoint, the visual object (431) (e.g., a line having start / end arrows) may be displayed between the user (410) wearing the wearable device (103) (or the wearable device (103)) and the external electronic device (421). The screen may include an extended reality (e.g., virtual reality, augmented reality, or mixed reality) provided through the wearable device (103). Alternatively, for example, if the screen displayed through the display is a first-person view, the visual object (431) may display a visual object (431) extending from the external electronic device (421) to the point where the screen begins. While FIG. 4A illustrates the third-person view, the embodiments of the present disclosure are not limited thereto. In addition, for example, the wearable device (103) may display another visual object (432) (e.g., a line with start / end arrows) indicating a connection established with the external electronic device (422).
[0092] In the embodiment (400) of FIG. 4A, the visual object (431) and the visual object (432) illustrate visual objects representing a connection, but the embodiments of the present disclosure are not limited thereto. For example, the visual object (431) and the visual object (432) may include information about the connection. For example, the information about the connection may include whether data is being transmitted and received by the connection, the time at which the connection was established, the type of the connection (e.g., BT, Wi-Fi), the communication status of the connection (e.g., signal quality), or the subject of the connection (e.g., information for distinguishing between a pairing device and a connection device in the case of BT).
[0093] According to one embodiment, the wearable device (103) may further display a visual object indicating a connection status between external electronic devices. For example, the wearable device (103) may display a visual object (433) (e.g., a dotted line) indicating a connection between external electronic devices (421) and external electronic devices (422). According to one embodiment, the visual object (433) indicating the connection between the external electronic devices may have a different shape from the visual object (431) (or visual object (432)) indicating the connection between the wearable device (103) and the external electronic device. However, the embodiments of the present disclosure are not limited thereto. For example, the visual object (433) and the visual object (431) (or visual object (432)) may have the same shape. The shape of the visual object indicating the connection status may be set by the user (410). Specific details related thereto are described below in FIG. 11.
[0094] Additionally, the wearable device (103) may further display an indicator (434) (e.g., a dotted line) indicating that a connection is possible based on identifying that a connection can be established with the external electronic device (423). For example, the indicator (434) may represent a visual object indicating that a connection is possible between the external electronic device (423) and the wearable device (103). According to one embodiment, the indicator (434) may have a different shape from the visual object (431) (or visual object (432)) indicating the connection.
[0095] In one embodiment, the connection and the possibility of connection may be included in the connection state. However, the embodiments of the present disclosure are not limited thereto. For example, the connection state may include a state in which the connection is established but no data is transmitted or received, or a state in which there is a record of a connection being established in the past.
[0096] FIG. 4b illustrates an example of a method for displaying a visual object indicating a connection status between external electronic devices.
[0097] The wearable device (103) of FIG. 4B may represent an example of the electronic device (101) of FIG. 1. FIG. 4B illustrates an embodiment (405) of a connection between a plurality of external electronic devices (421, 422, 423).
[0098] Referring to embodiment (405), the wearable device (103) can identify multiple external electronic devices (421, 422, 423). Specific details related thereto are substantially the same as those in FIG. 4A, and thus, redundant descriptions are omitted.
[0099] Referring to embodiment (405), a connection may be established between a plurality of external electronic devices (421, 422, 423). For example, a connection may be established between an external electronic device (421) and an external electronic wearable device (422). Also, for example, a connection may be established between an external electronic device (422) and an external electronic device (423). In the example of FIG. 4B, a connection may not be established between the wearable device (103) and the plurality of external electronic devices (421, 422, 423). However, the embodiments of the present disclosure are not limited thereto. Referring to embodiment (405) of FIG. 4B, the external electronic device (422) may be connected to both the external electronic device (421) and the external electronic device (423). The external electronic device (422) may be referred to as an external electronic device that supports multi-control. Specific details related to this are described in FIGS. 8a and 8b below.
[0100] According to one embodiment, the wearable device (103) may display a visual object indicating a connection status. For example, the wearable device (103) may display a visual object (441) indicating a connection established between an external electronic device (421) and an external electronic device (422). For example, the wearable device (103) may display a visual object (441) indicating a connection established between an external electronic device (421) and an external electronic device (422) through a display (e.g., the display module (160) of FIG. 1). Additionally, for example, the wearable device (103) may display a visual object (442) indicating a connection established between an external electronic device (422) and an external electronic device (423).
[0101] In the embodiment (405) of FIG. 4B, the visual object (441) and the visual object (442) illustrate visual objects representing a connection, but the embodiments of the present disclosure are not limited thereto. For example, the visual object (441) and the visual object (442) may include information about the connection. For example, the information about the connection may include whether data is being transmitted and received by the connection, the time at which the connection was established, the type of the connection (e.g., BT, Wi-Fi), the communication status of the connection (e.g., signal quality), or the subject of the connection (e.g., information for distinguishing between a pairing device and a connection device in the case of BT).
[0102] Referring to FIGS. 4A and 4B , the wearable device (103) may display a visual object indicating the connection status between external electronic devices or the connection status between the external electronic device and the wearable device (103) through the display. In addition, the wearable device (103) may change the connection status based on identifying a user's (410) input to the visual object (e.g., visual object (441), visual object (442)). For example, the change in the connection status may include switching, releasing (or disconnecting), or establishing the connection. Accordingly, the electronic device and method according to embodiments of the present disclosure may improve usability in an XR environment and provide an immersive user experience to the user by processing the user's input.
[0103] Figure 5 illustrates an exemplary block diagram of a wearable device within a network environment.
[0104] The wearable device (103) of FIG. 5 may represent an example of the electronic device (101) of FIG. 1, the wearable device (103) of FIGS. 2A to 3B, and the wearable device (103) of FIGS. 4A and 4B.
[0105] Referring to FIG. 5, an exemplary situation is illustrated in which a wearable device (103), a server (530), and an external electronic device (540) are connected to each other based on a wired network and / or a wireless network. For example, the wired network may include a network such as the Internet, a local area network (LAN), a wide area network (WAN), or a combination thereof. For example, the wireless network may include a network such as long term evolution (LTE), 5g new radio (NR), wireless fidelity (Wi-Fi), Zigbee, near field communication (NFC), Bluetooth, Bluetooth low-energy (BLE), or a combination thereof. Although the wearable device (103), the server (530), and the external electronic device (540) are illustrated as being directly connected, the wearable device (103), the server (530), and the external electronic device (540) may be indirectly connected via one or more routers and / or access points (APs). In other words, the wearable device (103) is exemplified as being directly connected to the communication circuit of the server (530) (or external electronic device (540)) via the communication circuit (509), but the embodiments of the present disclosure are not limited thereto.
[0106] Referring to FIG. 5, according to one embodiment, a wearable device (103) may include at least one of a processor (501), a display (503), a sensor (505), a camera (507), a communication circuit (509), and a memory (511). The processor (501), the display (503), the sensor (505), the camera (507), the communication circuit (509), and the memory (511) may be electronically and / or operably coupled with each other by a communication bus. Hereinafter, operably coupled hardware components may mean that a direct connection or an indirect connection is established between the hardware components, either wired or wireless, such that a second hardware component is controlled by a first hardware component among the hardware components. Although illustrated based on different blocks, the embodiment is not limited thereto, and some of the hardware components illustrated in FIG. 5 (at least a portion of the processor (501), the communication circuit (509), and the memory (511)) may be included in a single integrated circuit such as a system on a chip (SoC). The type and / or number of hardware components included in the wearable device (103) is not limited to those illustrated in FIG. 5. For example, the wearable device (103) may include only some of the hardware components illustrated in FIG. 5.
[0107] According to one embodiment, the processor (501) of the wearable device (103) may include a hardware component for processing data based on one or more instructions. The hardware component for processing data may include, for example, an arithmetic and logic unit (ALU), a floating point unit (FPU), and a field programmable gate array (FPGA). As an example, the hardware component for processing data may include a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processing (DSP), and / or a neural processing unit (NPU). The processor (501) may be or correspond to one or more processors. For example, the processor (501) may have a multi-core processor structure, such as a dual core, a quad core, or a hexa core. The processor (501) of FIG. 5 may include at least a part of the processor (120) of FIG. 1. In some embodiments, the processor (501) may correspond to at least one processor, such as a CPU, a microprocessor unit (MPU), an application processor (AP), a coprocessor (CP), a system-on-chip (SoC), or an integrated circuit (IC).
[0108] According to one embodiment, a display (503) of a wearable device (103) can output visualized information (e.g., visual objects of FIGS. 4A, 4B, 8A, 8B, 9, 10A, 10B, and 11) to a user. The number of displays (503) included in the wearable device (103) may be one or more. For example, the display (503) may be controlled by a processor (501) and / or a graphic processing unit (GPU) to output visualized information to a user. The display (503) may include a flat panel display (FPD) and / or electronic paper. The FPD may include a liquid crystal display (LCD), a plasma display panel (PDP), a digital mirror device (DMD), one or more light emitting diodes (LEDs), and / or a micro LED. The above LED may include an OLED (organic LED). The display (503) of FIG. 5 may include at least a portion of the display module (160) of FIG. 1.
[0109] In one embodiment, light transmission may occur through at least a portion of the display (503). The wearable device (103) may provide a user experience related to augmented reality by providing a combination of light output through the display (503) and light transmitted through the display (503) to the user. As described above with reference to FIGS. 2A and 2B and / or 3A and 3B , the display (503) of the wearable device (103) according to one embodiment may have a structure for covering the entire FoV of the user, or radiating light toward the FoV of the user, while being worn on a body part of the user, such as the head. The wearable device (103) may include other output means for outputting information in a form other than a visual or auditory form. For example, the wearable device (103) may include at least one speaker for outputting an audio signal, and / or a motor (or actuator) for providing haptic feedback based on vibration. In the above example, a display (503) for providing a user experience related to the augmented reality is exemplified, but the embodiments of the present disclosure are not limited thereto. For example, the wearable device (103) may render an image for providing a user experience related to virtual reality, and display the rendered image through the display (503).
[0110] According to one embodiment, the wearable device (103) may include a sensor (505) and a camera (507). For example, the wearable device (103) may include a sensor (505) for identifying a designated event, another designated event, and a user's input. For example, the sensor (505) may include an IMU (or IMU sensor), a biometric sensor, an acoustic sensor (or a microphone (or mic)), a gyro sensor, a gravity sensor, and / or an acceleration sensor. For example, the wearable device (103) may include a camera (507) (or an image sensor) for recognizing a real environment. The sensor (505) of FIG. 5 may include at least a portion of the sensor module (176) of FIG. 1. The camera (507) of FIG. 5 may include at least a portion of the camera module (180) of FIG. 1. For example, the wearable device (103) may identify the location of external electronic devices (521, 522, 523) through a sensor (505) and / or a camera (507).
[0111] According to one embodiment, the communication circuit (509) of the wearable device (103) may include hardware for supporting transmission and / or reception of electrical signals between external electronic devices (520) that are targets of identification and the wearable device (103). In addition, the communication circuit (509) of the wearable device (103) may include hardware for supporting transmission and / or reception of electrical signals between a server (530) providing an event and an external electronic device (540). The communication circuit (509) may include, for example, at least one of a modem (MODEM), an antenna, and an optical / electronic (O / E) converter. The communication circuit (509) may support transmission and / or reception of electrical signals based on various types of communication means, such as Ethernet, Bluetooth (BT), Bluetooth low energy (BLE), ZigBee, long term evolution (LTE), 5G new radio (NR), and wireless-fidelity (Wi-Fi). The communication circuit (509) of FIG. 5 may include at least a portion of the communication module (190) and / or antenna module (197) of FIG. 1.
[0112] According to one embodiment, the memory (511) of the wearable device (103) may include a hardware component for storing data and / or instructions input to and / or output from the processor (501). The memory (511) may include, for example, a volatile memory such as a random-access memory (RAM), and / or a non-volatile memory such as a read-only memory (ROM). The volatile memory may include, for example, at least one of a dynamic RAM (DRAM), a static RAM (SRAM), a cache RAM, and a pseudo SRAM (PSRAM). The non-volatile memory may include, for example, at least one of a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), a flash memory, a hard disk, a compact disc, and an embedded multimedia card (eMMC). The memory (511) of FIG. 5 may include at least a part of the memory (130) of FIG. 1.
[0113] Additionally, according to one embodiment, the wearable device (103) may include an output means for outputting information in a form other than a visualized form. For example, the wearable device (103) may include a speaker for outputting an acoustic signal. For example, the wearable device (103) may include a motor for providing haptic feedback based on vibration.
[0114] Referring to FIG. 5, according to one embodiment, one or more instructions (or commands) representing operations and / or actions to be performed on data by a processor (501) of the wearable device (103) may be stored in the memory (511) of the wearable device (103). A set of one or more instructions may be referred to as a program, firmware, an operating system, a process, a routine, a sub-routine, and / or an application. Hereinafter, when an application is installed in an electronic device (e.g., a wearable device (103)), it may mean that one or more instructions provided in the form of an application are stored in the memory (511), and that the one or more applications are stored in a format executable by the processor of the electronic device (e.g., a file having an extension designated by the operating system of the wearable device (103)). According to one embodiment, the wearable device (103) may execute one or more instructions stored in the memory (511) to perform the operations of FIGS. 6A and 6B, FIG. 12, and FIG. 13.
[0115] According to one embodiment, the wearable device (103) can identify external electronic devices (520). For example, the wearable device (103) can identify the external electronic devices (520) based on the communication circuit (509). For example, the external electronic devices (520) can include at least one of an electronic device connectable via short-range wireless communication based on the communication circuit (509), an electronic device having the same account information as the account information of the wearable device (103) identified based on the communication circuit (509), an electronic device within a specified distance from the wearable device (103) identified based on (or identified using) the communication circuit (509), or an electronic device connected to the same access point (AP) as the wearable device (103) identified based on the communication circuit (509).
[0116] According to one embodiment, at least some of the external electronic devices (520) may be associated with a visual object indicating a connection status through the display (503) of the wearable device (103). For example, at least some of the external electronic devices (520) may indicate a target (or an identification target) on which the visual object is to be displayed.
[0117] According to one embodiment, the wearable device (103) can identify an external electronic device within the FoV (515) of the wearable device (103) among the external electronic devices (520). For example, the wearable device (103) can identify an external electronic device (521) and an external electronic device (522). For example, the wearable device (103) can provide the external electronic device (521) and the external electronic device (522) within the FoV (515) through the display (503). For example, the wearable device (103) that provides virtual reality can display a rendered image representing the external electronic device (521) and a rendered image representing the external electronic device (522) through the display (503). Alternatively, for example, a wearable device (103) providing augmented reality may provide an external electronic device (521) and an external electronic device (522) that are transmitted through a display (503). The direction in which the FoV (515) faces may change depending on the gaze direction of a user wearing the wearable device (103). The FoV (515) may be referred to as a gaze direction of the user of the wearable device (103), a recognition direction, a recognition range, or an identification range of the wearable device (103). The wearable device (103) identifying external electronic devices based on the communication circuit (509) and identifying some external electronic devices within the FoV (515) among the identified external electronic devices is described in detail in FIG. 7 below.
[0118] According to one embodiment, the wearable device (103) can identify the connection status of external electronic devices (520). For example, the wearable device (103) can transmit a first signal to each of the external electronic devices (520). For example, the wearable device (103) can broadcast or multicast the first signal to each of the external electronic devices (520). According to one embodiment, the wearable device (103) can receive a second signal from at least some of the external electronic devices (520) in response to the first signal. For example, each of the first signal and the second signal can be referred to as an advertising signal, an identification signal, a broadcasting signal, and a multicasting signal. According to one embodiment, the wearable device (103) can establish a connection with at least some of the external electronic devices or identify the connection status of at least some of the external electronic devices based on the second signal. For example, the wearable device (103) may transmit the first signal at a specified interval or based on at least one of a user's input.
[0119] According to one embodiment, the first signal may include at least one of identification information of the wearable device (103), account information of the user of the wearable device (103), connection status information of the wearable device (103), connection record information of the wearable device (103), transmission intensity information of the first signal, detection area information based on the first signal, battery information of the wearable device (103), or voice output information of the wearable device (103). In addition, according to one embodiment, the second signal may include at least one of identification information of the external electronic device, account information of the user of the external electronic device, connection status information of the external electronic device, connection record information of the external electronic device, transmission intensity information of the second signal, detection area information based on the second signal, battery information of the external electronic device, or sound output information of the external electronic device. The external electronic device may be included in at least some of the external electronic devices (520).
[0120] For example, the identification information may include an identifier of the wearable device (103) (or the external electronic device). The identifier may include address information or port information for connection. For example, the account information may include account information of the user registered in the wearable device (103). For example, the connection status information may include information about an electronic device to which the wearable device (103) (or the external electronic device) is currently connected (or paired). The connection status information may be referred to as pairing information or pairing status information. For example, the connection record information may include information about an electronic device to which the wearable device (103) (or the external electronic device) has a history of connection. The connection record information may be referred to as a pairing list, a connection history list, a connectable list, or a connection list. For example, the transmission intensity information may include the transmission intensity of a signal (e.g., the first signal or the second signal) transmitted by the wearable device (103) (or the external electronic device). For example, the detection area information may include information on an identifiable area based on the signal. For example, the battery information may include the current remaining battery level of the wearable device (103) (or the external electronic device). For example, the sound output information may include information on the role (e.g., left output device or right output device) of an output device included in the wearable device (103) (or the external electronic device).
[0121] According to one embodiment, the wearable device (103) may identify an event related to a connection with respect to an external electronic device that establishes a connection with the wearable device (103) or whose connection status the wearable device (103) recognizes. For example, the event related to the connection may include at least one of an input for a visual object for establishing a connection displayed through the display (503), receiving information from an external electronic device (e.g., external electronic devices (520)) or an external electronic device (e.g., a server (530) or an external electronic device (540)) connected to the wearable device (103) to determine whether a connection to the wearable device (103) is possible, executing a function for performing multiple connections, or an input for changing a connection.
[0122] According to one embodiment, the wearable device (103) may be connected to an external electronic device that provides an event related to the connection. For example, the wearable device (103) may be connected to a server (530) and an external electronic device (540). For example, the wearable device (103) may directly receive the event from the server (530). Alternatively, for example, the wearable device (103) may directly receive the event from the external electronic device (540). Alternatively, for example, the wearable device (103) may receive the event identified from the external electronic device (540) through the server (530). Alternatively, for example, the wearable device (103) may receive the event identified from the server (530) through the external electronic device (540). At this time, the server (530) and the external electronic device (540) may be connected to each other. For example, a server (530) and an external electronic device (540) that provide events related to the above connection may be referred to as source devices.
[0123] For example, the server (530) may represent an electronic device that is connected to external electronic devices (520) and manages the external electronic devices (520). For example, the external electronic devices (520) may provide information about changes in the connection status of each of the external electronic devices (520) (e.g., connection switching, establishment (or re-establishment), or release) to the server (530). The server (530) may provide the information to the wearable device (103). For example, the external electronic device (540) may include an electronic device in which the same account information as the wearable device (103) is registered. For example, the account information may include the account information of the user of the wearable device (103).
[0124] As described above, the wearable device (103) is described as acquiring the event from the server (530) or the external electronic device (540), but the embodiments of the present disclosure are not limited thereto. For example, the wearable device (103) may acquire the event directly from an external electronic device connected to the wearable device (103) among the external electronic devices (520). Alternatively, for example, the wearable device (103) may directly identify the event for the connected external electronic device.
[0125] According to one embodiment, the wearable device (103) may display a visual object representing the connection via the display (503) based on identifying the event associated with the connection. For example, the visual object may include visual information (or an image) for representing the connection. Specific details related thereto are described below in FIG. 11.
[0126] With reference to the above, examples of connection states and visual indications according to connection states according to embodiments of the present disclosure are as follows.
[0127] For example, a wearable device (103) may be connected to an external electronic device (521). The wearable device (103) may display a visual object indicating a connection between the wearable device (103) and the external electronic device (521) and information about the connection. For example, the information about the connection may include whether data is being transmitted and received through the connection, the time the connection was established, the type of the connection (e.g., BT, Wi-Fi), the communication status of the connection (e.g., signal quality), or the subject of the connection (e.g., information for distinguishing between a pairing device and a connection device in the case of BT).
[0128] For example, a connection may be established between an external electronic device (521) and an external electronic device (522). The wearable device (103) may display a visual object indicating the connection between the external electronic device (521) and the external electronic device (522) and information about the connection.
[0129] For example, a connection may be established between an external electronic device (521) and an external electronic device (523). The wearable device (103) may display a visual object indicating the connection between the external electronic device (521) and the external electronic device (523) and information about the connection. Alternatively, for example, a connection may be established between an external electronic device (523) and a wearable device (103). The wearable device (103) may display a visual object indicating the connection between the wearable device (103) and the external electronic device (523) and information about the connection.
[0130] In the examples described above, the wearable device (103) may receive information about the connection from external electronic devices (521, 522, 523), or may receive information through an external electronic device (540) or a server (530).
[0131] Figures 6a and 6b illustrate examples of a method for displaying a visual object indicating the connection status of an external electronic device.
[0132] FIG. 6A illustrates that at least some of the above methods may be performed by the wearable device (103) of FIG. 5 . For example, at least some of the above methods may be controlled by the processor (501) of the wearable device (103). In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.
[0133] In operation (600), the wearable device (103) may confirm a request for displaying communication information. For example, the request for displaying the communication information may include at least one of: an input for a visual object for establishing a connection displayed through the display (503); receiving information for confirming whether a connection to an external electronic device (e.g., external electronic devices (520) of FIG. 5) or the wearable device (103) is possible from an external electronic device (e.g., a server (530) or an external electronic device (540) of FIG. 5) connected to the wearable device (103); executing a function for performing multiple connections; or an input for changing a connection.
[0134] For example, receiving information to determine whether the connection is possible may indicate that an external electronic device (e.g., a server (530) or an external electronic device (540) of FIG. 5) that has identified whether any external electronic device is capable of connection with the external electronic device (e.g., external electronic devices (520) of FIG. 5) or the wearable device (103) provides the information. For example, receiving the information may include the wearable device (103) obtaining the information from the identified external electronic device by performing a scan.
[0135] For example, the function for performing the above multiple connections may include multi-control. However, the embodiments of the present disclosure are not limited thereto. For example, the function for performing the above multiple connections may be understood to include a designated function or service related to the connection. For example, the multiple connections may include connections of multiple electronic devices supporting BT when using BT. For example, the multiple connections may include connections of multiple audio output devices and devices providing audio information. For example, the function for performing the above multiple connections may include execution of quick share.
[0136] For example, the change in the connection status may include switching the connection, releasing the connection, or establishing (or re-establishing) the connection. The change in the connection status may be identified directly by the wearable device (103), identified based on information obtained from an external electronic device connected to the wearable device (103), or identified based on information obtained by a server connected to the wearable device (103) from an external electronic device connected to the server.
[0137] In operation (605), the wearable device (103) may determine whether communication is connected based on the request. For example, the wearable device (103) may identify one or more external electronic devices. For example, the wearable device (103) may identify the one or more external electronic devices using short-range communication based on the communication circuit (509). For example, the short-range communication may include BT (Bluetooth), BLE (Bluetooth low energy), Wi-Fi (Wireless fidelity), or UWB (ultra-wideband). However, the embodiments of the present disclosure are not limited thereto. For example, the wearable device (103) may also identify the one or more external electronic devices based on a wireless communication or wired communication technology that is different from the short-range communication. The operation of identifying the plurality of external electronic devices may be referred to as searching, discovering, detecting, or scanning the one or more external electronic devices. According to one embodiment, the wearable device (103) may identify one or more external electronic devices within the FoV of the wearable device (103) among the one or more external electronic devices. The direction of the FoV may change depending on the gaze direction of the user wearing the wearable device (103).
[0138] According to one embodiment, the wearable device (103) may, in response to the request, determine whether a connection has been established with one or more external electronic devices.
[0139] In operation (610), if the connection is established, the wearable device (103) may display communication information related to the connection. For example, the communication information may include a visual object representing the connection. For example, the visual object may include a line representing a connection between at least some of the one or more external electronic devices with which the connection is established and the wearable device (103). In addition, for example, the communication information may include whether data is being transmitted and received by the connection, the time at which the connection is established, the type of the connection (e.g., BT, Wi-Fi), the communication status of the connection (e.g., signal quality), or the subject of the connection (e.g., information for distinguishing between a pairing device and a connection device in the case of BT).
[0140] In operation (615), the wearable device (103) may display communication information related to connection possibility if the connection is not established. For example, the communication information related to connection possibility may include an indicator indicating connection possibility, visual information indicating the connection progress status, or visual information indicating a connection failure.
[0141] For example, if the connection with one or more external electronic devices is not established, the wearable device (103) may display an indicator indicating that connection is possible for an external electronic device among the one or more external electronic devices that is connectable. For example, the indicator may be included in the visual information.
[0142] Additionally, for example, the wearable device (103) may display visual information indicating a connection progress status or visual information indicating a connection failure based on identifying a request related to the connection. For example, when attempting to connect to an external electronic device, if the external electronic device is connected to another external electronic device and connection is impossible, the wearable device (103) may display visual information indicating connection information between the external electronic device and the other external electronic device. Alternatively, the wearable device (103) may display visual information indicating a connection progress status between the wearable device (103) and the external electronic device, and then display visual information indicating a failure.
[0143] FIG. 6b illustrates that at least some of the above methods may be performed by the wearable device (103) of FIG. 5 . For example, at least some of the above methods may be controlled by the processor (501) of the wearable device (103). In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.
[0144] In operation (650), the wearable device (103) may identify one or more external electronic devices. For example, the wearable device (103) may identify the one or more external electronic devices based on the communication circuit (509). For example, the wearable device (103) may identify the one or more external electronic devices using short-range communication based on the communication circuit (509). For example, the short-range communication may include Bluetooth (BT), Bluetooth low energy (BLE), Wireless fidelity (Wi-Fi), or Ultra-Wideband (UWB). However, the embodiments of the present disclosure are not limited thereto. For example, the wearable device (103) may also identify the one or more external electronic devices based on a wireless communication or wired communication technology that is different from the short-range communication. The act of identifying the plurality of external electronic devices may be referred to as searching, discovering, detecting, or scanning the one or more external electronic devices.
[0145] According to one embodiment, the one or more external electronic devices may include at least one of an electronic device connectable via short-range wireless communication based on the communication circuit (509), an electronic device having the same account information as the account information of the wearable device (103) identified based on the communication circuit (509), an electronic device within a specified distance from the wearable device (103) identified based on the communication circuit (509), or an electronic device connected to the same access point (AP) as the wearable device (103) identified based on the communication circuit (509).
[0146] For example, the one or more external electronic devices may include electronic devices connected to the same node (e.g., AP (access point)) as the wearable device (103). For example, multiple APs may be deployed within a specific area to provide location-based services. The deployment of the multiple APs may be configured according to the range of a service area or the coverage of the APs. In addition, a server (e.g., WPS (Wi-Fi Positioning System)) that manages APs may manage information on the multiple APs and location information on the multiple APs. For example, the wearable device (103) may identify the one or more external electronic devices connected to the same AP as the wearable device (103) based on the information on the multiple APs and the location information on the multiple APs obtained from the server.
[0147] For example, the one or more external electronic devices may be identified based on the strength of a signal received by the wearable device (103). For example, the wearable device (103) may estimate a distance based on the strength and pattern of the received signal for indoor positioning. For example, the strength of the signal may include a received signal strength indicator (RSSI). For example, the RSSI may be an indicator representing the strength of a signal (unit: dBm), where the strength of the signal decreases with distance. The RSSI may be defined as in the following mathematical equation.
[0148]
[0149] The above n may represent an attenuation coefficient, the above A may represent an RSSI value measured at a specific location (e.g., 1 m (meter)) as a reference value, and the above d may represent a distance from a location where a signal is transmitted. For example, the attenuation constant may change depending on the environment in which the signal is transmitted and received. For example, the wearable device (103) may estimate a distance value based on the RSSI. The wearable device (103) may identify one or more external electronic devices within a specified distance of the estimated distance value.
[0150] For example, the one or more external electronic devices can be identified based on a location positioning technique (e.g., Wi-Fi-based location positioning, BLE-based location positioning, IoT (internet of things) server-based location positioning). For example, the wearable device (103) can identify distance information and direction information regarding the external electronic device through reception and transmission of high-frequency signals defined in 802.11.ad of the Wi-Fi communication standard. Based on the distance information and the direction information, the AP connected to the external electronic device can identify the exact location of the electronic device. For example, the wearable device (103) can identify the one or more external electronic devices based on the distance information and the direction information acquired from the AP. In addition, for example, the wearable device (103) can identify the one or more external electronic devices based on the angle of arrival (AoA) and angle of departure (AoD) identified using BLE. For example, the wearable device can identify the one or more external electronic devices based on the AoA and AoD. In addition, for example, the wearable device (103) can identify the one or more external electronic devices based on information obtained from a server that manages IoT devices (e.g., server (530) of FIG. 5). For example, the server can pre-designate or identify the locations of the one or more external electronic devices, which are IoT devices connected to the server. For example, the wearable device (103) can identify the one or more external electronic devices based on the server.
[0151] Additionally, for example, the one or more external electronic devices may cause the wearable device (103) to identify the one or more external electronic devices by providing a signal including information that identifies the wearable device (103) to a server (e.g., an IoT server). For example, the one or more external electronic devices may identify a surrounding object based on a camera or a sensor. For example, a UWB sensor may be used to identify the object. In this case, the object may include a wearable device (103) identified based on the camera or the sensor. The external object may provide information about the identified wearable device (103) to the server. For example, the one or more external electronic devices may display a user interface through a display. For example, the user interface may be a user interface for the user to confirm whether the device (e.g., wearable device (103)) to be identified through the server (or P2P (peer-to-peer)) is correct. Based on the user's input (or gesture) to the user interface, the one or more external electronic devices may provide information to the server. The server may perform a connection (or mapping) between the wearable device (103) and the one or more external electronic devices based on the information. At this time, the connection may be accompanied by authentication (e.g., a predetermined pattern) to enhance security. Accordingly, the wearable device (103) may also identify the one or more external electronic devices. In the above example, the case where the object is the wearable device (103) is exemplified, but the embodiments of the present disclosure are not limited thereto.For example, by identifying a user of a designated gesture or wearable device (103), the one or more external electronic devices can identify the presence of the wearable device (103) and thereby provide the information about the wearable device (103) to the server. The wearable device (103) can also identify the one or more external electronic devices based on the information provided from the server. According to one embodiment, the wearable device (103) can use at least one of the methods for identifying the location of the one or more external electronic devices. Furthermore, embodiments of the present disclosure are not limited thereto, and other methods for identifying the location of the external object can be used.
[0152] According to one embodiment, the wearable device (103) can identify one or more external electronic devices within the FoV of the wearable device (103) among the one or more external electronic devices. The direction of the FoV can be changed depending on the gaze direction of the user wearing the wearable device (103).
[0153] According to one embodiment, the wearable device (103) can identify, among the one or more external electronic devices within the FoV, devices that include the same account information as the wearable device (103). At this time, if the shapes of the devices including the same account information are similar, the wearable device (103) can distinguish the devices through additional signaling. For example, the wearable device (103) can transmit a scanning signal to the devices. Within the devices, the distinction between the devices can be performed through a screen (or UI) displayed based on the scanning signal. Alternatively, for example, a device among the devices that is not directly connected to the wearable device (103) can be distinguished through indirect signaling via another external electronic device (e.g., a server).
[0154] As described above, the wearable device (103) can identify the plurality of connectable external electronic devices and perform differentiation of the plurality of external electronic devices through FoV, account information, and additional signaling.
[0155] In operation (655), the wearable device (103) may identify a first connection. For example, the first connection may include a connection between one or more external electronic devices or a connection between some of the one or more external electronic devices and the wearable device (103).
[0156] According to one embodiment, the wearable device (103) may identify a connection status of the one or more external electronic devices. For example, the wearable device (103) may transmit a first signal to each of the one or more external electronic devices. For example, the wearable device (103) may broadcast or multicast the first signal to each of the one or more external electronic devices. According to one embodiment, the wearable device (103) may receive a second signal from at least some of the one or more external electronic devices in response to the first signal. For example, each of the first signal and the second signal may be referred to as an advertising signal, an identification signal, a broadcasting signal, or a multicasting signal. According to one embodiment, the wearable device (103) may establish a connection with at least some of the external electronic devices or identify the connection status of at least some of the external electronic devices based on the second signal. For example, the wearable device (103) may transmit the first signal at a specified interval or based on at least one of a user's input.
[0157] According to one embodiment, the first signal may include at least one of identification information of the wearable device (103), account information of the user of the wearable device (103), connection status information of the wearable device (103), connection record information of the wearable device (103), transmission intensity information of the first signal, detection area information based on the first signal, battery information of the wearable device (103), or audio output information of the wearable device (103). In addition, according to one embodiment, the second signal may include at least one of identification information of the external electronic device, account information of the user of the external electronic device, connection status information of the external electronic device, connection record information of the external electronic device, transmission intensity information of the second signal, detection area information based on the second signal, battery information of the external electronic device, or audio output information of the external electronic device. The external electronic device may be included in at least a part of the one or more external electronic devices.
[0158] For example, the identification information may include an identifier of the wearable device (103) (or the external electronic device). The identifier may include address information or port information for connection. For example, the account information may include account information of the user registered in the wearable device (103). For example, the connection status information may include information about an electronic device to which the wearable device (103) (or the external electronic device) is currently connected (or paired). The connection status information may be referred to as pairing information or pairing status information. For example, the connection record information may include information about an electronic device to which the wearable device (103) (or the external electronic device) has a history of connection. The connection record information may be referred to as a pairing list, a connection history list, a connectable list, or a connection list. For example, the transmission intensity information may include the transmission intensity of a signal (e.g., the first signal or the second signal) transmitted by the wearable device (103) (or the external electronic device). For example, the detection area information may include information on an identifiable area based on the signal. For example, the battery information may include the current remaining battery level of the wearable device (103) (or the external electronic device). For example, the sound output information may include information on the role (e.g., left output device or right output device) of an output device included in the wearable device (103) (or the external electronic device).
[0159] In operation (660), the wearable device (103) may display a first visual object based on identifying an event related to the first connection. For example, the wearable device (103) may obtain the event from another external electronic device. For example, the other external electronic device may include the server (530) or the external electronic device (540) of FIG. 5. Alternatively, for example, the wearable device (103) may obtain the event from an external electronic device that has established the first connection with the wearable device (103), or the wearable device (103) may identify the event for the first connection. For example, the wearable device (103) may generate (or render) the first visual object based on identifying the event related to the first connection, and display the generated first visual object.
[0160] For example, an event related to the first connection may include at least one of an input for a visual object for establishing a connection displayed through a display (503), receiving information for confirming whether a connection to an external electronic device (e.g., external electronic devices (520) of FIG. 5) or a wearable device (103) is possible from an external electronic device (e.g., a server (530) or an external electronic device (540) of FIG. 5) connected to the wearable device (103), executing a function for performing multiple connections, or an input for changing a connection.
[0161] For example, the visual object for establishing the connection may include an icon indicating a function (or menu) for checking the communication status with the electronic device or establishing a connection via a communication technique. For example, the function (or menu) may include a connection according to a communication technique such as Bluetooth, Wi-Fi, or cellular. For example, the icon may be included in the settings of the electronic device. The electronic device may include a wearable device (103). For example, the visual object may be used when the wearable device (103) checks a device that is connectable to the surroundings or to check currently connected information.
[0162] For example, receiving information to determine whether the connection is possible may indicate that an external electronic device (e.g., a server (530) or an external electronic device (540) of FIG. 5) that has identified whether any external electronic device is capable of connection with the external electronic device (e.g., external electronic devices (520) of FIG. 5) or the wearable device (103) provides the information. For example, receiving the information may include the wearable device (103) obtaining the information from the identified external electronic device by performing a scan.
[0163] For example, the function for performing the above multiple connections may include multi-control. However, the embodiments of the present disclosure are not limited thereto. For example, the function for performing the above multiple connections may be understood to include a designated function or service related to the connection. For example, the multiple connections may include connections of multiple electronic devices supporting BT when using BT. For example, the multiple connections may include connections of multiple audio output devices and devices providing audio information. For example, the function for performing the above multiple connections may include execution of quick share.
[0164] For example, the action of changing the connection state may include the first connection being switched to another connection (e.g., a second connection below), the first connection being released, or the first connection being established (or re-established).
[0165] According to one embodiment, the wearable device (103) may display the first visual object based on identifying (or receiving) the event. For example, the first visual object may include a visual object for indicating the first connection. For example, the wearable device (103) may display the first visual object for a specified length of time from the time at which the event is identified. Alternatively, for example, the wearable device (103) may display the first visual object when the user's movement of the wearable device (103) is within a specified range. The movement may include a movement of a body part (e.g., a hand) of the user or a movement of the user. For example, the specified length of time and the specified range may be set by the user.
[0166] According to one embodiment, the first visual object may be displayed with respect to the electronic devices (e.g., the first electronic device and the second electronic device) that have established the first connection. The electronic devices may be included in the plurality of external electronic devices identified by the wearable device (103). For example, the first visual object may include a first icon representing connection to the first electronic device, a second icon representing connection to the second electronic device, and a line representing connection between the first icon and the second icon. In one example, when the first connection is established between the wearable device (103) and the external electronic device, the first electronic device or the second electronic device may include the wearable device (103). For example, when the electronic devices that have established the first connection are located within the FoV of the wearable device (103), the first visual object may extend from the first electronic device to the second electronic device. Alternatively, if the first electronic device among the electronic devices that established the first connection is located outside the FoV of the wearable device (103), the area corresponding to the location of the first electronic device within the FoV may extend to the second electronic device.
[0167] According to one embodiment, the wearable device (103) may further display an indicator indicating connection possibility together with the first visual object based on identifying an event related to the first connection. For example, the wearable device (103) may identify a connection possibility between external electronic devices or a connection possibility between the external electronic device and the wearable device (103) based on the first signal and the second signal. Alternatively, for example, the wearable device (103) may transmit a capability information inquiry message to the external electronic device and receive a capability information message in response thereto. The wearable device (103) may identify a connection possibility between external electronic devices or a connection possibility between the external electronic device and the wearable device (103) based on the capability information inquiry message and the capability information message. The wearable device (103) may, based on the identified connection possibility, display an indicator indicating the connection possibility between the external electronic device associated with the first connection and another external electronic device. The other external electronic device may represent an electronic device not associated with the first connection.
[0168] According to one embodiment, the wearable device (103) may display visual information indicating a connection progress status or visual information indicating a connection failure based on identifying an event related to the first connection. For example, when attempting to connect to an external electronic device, if the external electronic device is connected to another external electronic device and connection is impossible, the wearable device (103) may display visual information indicating connection information between the external electronic device and the other external electronic device. Alternatively, the wearable device (103) may display visual information indicating a connection progress status between the wearable device (103) and the external electronic device, and then display visual information indicating a failure.
[0169] In operation (665), the wearable device (103) may obtain an input for the first visual object. For example, the wearable device (103) may identify the user's input for the first visual object. For example, the user's input may include a drag (or drag-and-drop) input.
[0170] In one embodiment, the user input may cause a change in the first connection. For example, the change in the first connection may include switching from the first connection to another connection (e.g., a second connection below), disconnecting the first connection, or re-establishing the first connection. The following describes a switching from the first connection to another connection as an example, but the embodiments of the present disclosure are not limited thereto. For example, a case in which the first connection is disconnected is described in detail in FIG. 10b below.
[0171] In operation (670), the wearable device (103) may transmit a request signal for establishing a second connection. For example, the wearable device (103) may identify the second connection, which is changed from the first connection, based on identifying the acquisition of the input. If the electronic devices that established the first connection are a first electronic device and a second electronic device, the second connection may include a connection between the first electronic device and a third electronic device. In this case, the third electronic device may be identified based on the input. For example, the wearable device (103) may transmit the request signal to the first electronic device, the second electronic device, and the third electronic device. In this case, the request signal transmitted to the first electronic device may be used to request that the first connection be released (or disconnected) and the second connection be established. The request signal transmitted to the second electronic device may be used to request that the first connection be released. The request signal transmitted to the third electronic device may be used to request that the second connection be established. In one example, one of the first electronic device, the second electronic device, and the third electronic device may be a wearable device (103). In this case, the request signal may be transmitted only to the wearable device (103) and other electronic devices among the first electronic device, the second electronic device, and the third electronic device.
[0172] According to one embodiment, when the user's input is the drag input, the second connection may be established for executing a designated function based on the direction of the drag input. For example, when the direction of the drag input is a first direction from the first electronic device to the third electronic device, the second connection may be established for executing a first function. Conversely, when the direction of the drag input is a second direction from the third electronic device to the first electronic device, the second connection may be established for executing a second function different from the first function. In this case, the wearable device (103) may transmit the request signal for establishing the second connection for executing the first function or the second function. Specific details related thereto are described below in FIG. 9.
[0173] In operation (675), the wearable device (103) may receive a response signal. For example, the wearable device (103) may receive the response signal to the request signal. In the example, the response signal may indicate that the second connection is established. For example, the response signal may be received from the first electronic device or the third electronic device. Alternatively, for example, the response signal may be received from a server connected to the wearable device (103) and connected to the first electronic device or the third electronic device. The wearable device (103) may identify that the second connection is established based on receiving the response signal.
[0174] In operation (680), the wearable device (103) may display a second visual object indicating the second connection. For example, the wearable device (103) may stop displaying the first visual object and display the second visual object based on receiving the response signal. For example, the wearable device (103) may stop displaying the first visual object first and display the second visual object, or may display the second visual object first and stop displaying the first visual object. For example, the description of the second visual object may be substantially identical to the description of the first visual object.
[0175] According to one embodiment, the wearable device (103) may display information about the connection together with the first visual object and the second visual object indicating the connection. For example, the information about the connection may include whether data is being transmitted and received through the connection, the time the connection was established, the type of the connection (e.g., BT, Wi-Fi), the communication status of the connection (e.g., signal quality), or the subject of the connection (e.g., information for distinguishing between a pairing device and a connection device in the case of BT).
[0176] FIG. 7 illustrates an example of how a wearable device identifies multiple external electronic devices.
[0177] FIG. 7 illustrates an example of a method for a wearable device (103) to identify multiple external electronic devices based on a communication circuit and a FoV (730) of the wearable device (103). The wearable device (103) of FIG. 7 may include the wearable device (103) of FIG. 5. For example, the communication circuit may include a communication circuit (509).
[0178] Referring to FIG. 7, the wearable device (103) can identify one or more external electronic devices (711, 712, 713, 714, 715) based on the communication circuit (509). For example, the wearable device (103) can identify one or more external electronic devices (711, 712, 713, 714, 715) using short-range communication based on the communication circuit (509). However, the embodiments of the present disclosure are not limited thereto. For example, the wearable device (103) can also identify one or more external electronic devices (711, 712, 713, 714, 715) based on a wireless communication or wired communication technology that is different from the short-range communication.
[0179] According to one embodiment, the one or more external electronic devices (711, 712, 713, 714, 715) may include at least one of an electronic device connectable via short-range wireless communication using the communication circuit (509), an electronic device having the same account information as the account information of the wearable device (103) identified using the communication circuit (509), an electronic device within a specified distance from the wearable device (103) identified using the communication circuit (509), or an electronic device connected to the same access point (AP) as the wearable device (103) identified using the communication circuit (509). Referring to FIG. 7, the wearable device (103) may not be able to identify the external electronic device (720). For example, the external electronic device (720) may be different from the connectable electronic device, the electronic device having the same account information, the device within the specified distance, and the electronic device connected to the same AP.
[0180] According to one embodiment, the wearable device (103) can identify external electronic devices (711, 712) within the FoV (730) from among the identified one or more external electronic devices (711, 712, 713, 714, 715). For example, the wearable device (103) can identify external electronic devices (711, 712) to be displayed within the FoV (730) of the wearable device (103) from among the one or more external electronic devices (711, 712, 713, 714, 715) based on the communication circuit (509).
[0181] According to one embodiment, the wearable device (103) may display a visual object (503) indicating a connection status of at least some of the external electronic devices (711, 712).
[0182] According to one embodiment, the wearable device (103) can identify a device among the external electronic devices (711, 712) within the FoV (730) that includes the same account information as the wearable device (103). For example, the wearable device (103) can identify that the external electronic device (711) has the same account information as the wearable device (103) and that the external electronic device (712) has different account information than the wearable device (103). For example, if the external electronic devices (711, 712) have the same account information and it is difficult to distinguish between the external electronic devices (711, 712) based on their appearance, the wearable device (103) can distinguish between the devices through additional signaling. For example, the wearable device (103) can transmit a signal for scanning the devices. Within the external electronic devices (711, 712), device distinction can be performed through a screen (or UI) displayed based on the signal for the scan. The wearable device (103) can be referred to as an advertiser, and the external electronic devices (711, 712) can be referred to as scanning devices. Alternatively, for example, a device among the external electronic devices (711, 712) that is not directly connected to the wearable device (103) can be distinguished through indirect signaling via another external electronic device (e.g., a server). As described above, the wearable device (103) can identify the plurality of connectable external electronic devices and distinguish the plurality of external electronic devices through FoV, account information, and additional signaling.
[0183] FIGS. 8A and 8B illustrate examples of a method for displaying visual objects indicating the connection status of an external electronic device supporting multiple connections.
[0184] The wearable device (103) of FIGS. 8A and 8B may include the wearable device (103) of FIG. 5. The plurality of connections may include connections for a multi-control service. For example, the multi-control service may indicate a function of an electronic device connected to a plurality of electronic devices to control the plurality of electronic devices. In the examples (800, 805) of FIGS. 8A and 8B, an external electronic device (811) may support the multi-control service. The example (800) of FIG. 8A may indicate an example of a connection for the multi-control service.
[0185] Referring to example (800) of FIG. 8A, the wearable device (103) can display a plurality of external electronic devices (811, 812, 813, 814) within the FoV (807). For example, the wearable device (103) can display a plurality of external electronic devices (811, 812, 813, 814) within the FoV (807) through the display (503). In example (800), for convenience of explanation, visual objects (821, 822) indicating connection states between the plurality of external electronic devices (811, 812, 813) within the FoV (807) are shown as being displayed, but embodiments of the present disclosure are not limited thereto. For example, if some of the multiple external electronic devices (811, 812, 813, 814) are located outside the FoV (807), the wearable device (103) may not display some of them.
[0186] Referring to example (800), the wearable device (103) may display visual objects (821, 822) indicating a connection status of an external electronic device (811). For example, the wearable device (103) may display a visual object (821) indicating a first connection between the external electronic device (811) and the external electronic device (812). Additionally, the wearable device (103) may display a visual object (822) indicating a second connection between the external electronic device (811) and the external electronic device (813).
[0187] According to one embodiment, a user may use the multi-control to control an external electronic device (811) with which a second connection is established based on an input to an external electronic device (813), and may also control an external electronic device (812) with which a first connection is established with the external electronic device (811). Based on identifying the input to the external electronic device (813), visual objects (821, 822) may be displayed.
[0188] Referring to example (805) of FIG. 8B, the wearable device (103) may display another visual object representing a third connection based on identifying an input (830) for a visual object. For example, the wearable device (103) may identify an input (830) for a visual object (822) representing the second connection. For example, the input (830) for the visual object (822) may cause the establishment of the third connection between the external electronic device (813) and the external electronic device (814). For example, the wearable device (103) may transmit a request signal for establishing the third connection to the external electronic device (813) and the external electronic device (814). At this time, the request signal transmitted to the external electronic device (813) may also request the disconnection of the second connection. Additionally, for example, the wearable device (103) may transmit a request signal for disconnection of the second connection to the external electronic device (811). Thereafter, the wearable device (103) may receive a response signal to the request signal. For example, the wearable device (103) may receive the response signal from the external electronic device (813) or the external electronic device (814). For example, the response signal may indicate establishment of the third connection. In response to receiving the response signal, the wearable device (103) may display a visual object (823) indicating the third connection.
[0189] In examples (800, 805) of FIGS. 8A and 8B, visual objects (821, 822, 823) are shown being displayed, but the embodiments of the present disclosure are not limited thereto. For example, as in example (800) of FIG. 8A, the first connection and the second connection are established for the external electronic device (811) supporting the multi-control, but the visual objects (821, 822) may not be displayed. Thereafter, based on identifying an event related to the second connection (e.g., an input (830) for changing the second connection to the third connection), the visual objects (821, 822) may be displayed. Thereafter, when the connection state is switched from the second connection to the third connection based on the input (830), the visual objects (821, 823) may be displayed.
[0190] Figure 9 illustrates an example of a method for establishing a connection based on an input for a visual object indicating a connection status and displaying a visual object according to the established connection.
[0191] The wearable device (103) of FIG. 9 may represent an example of the wearable device (103) of FIG. 5. FIG. 9 illustrates an example (901) of displaying a visual object (921) indicating a first connection between the wearable device (103) and an external electronic device (911), and examples (902, 903) of displaying a visual object (922) or a visual object (923) indicating a second connection between the external electronic device (911) and the external electronic device (912).
[0192] Referring to example (901), the wearable device (103) may display a visual object (921) representing the first connection based on identifying an input to the external electronic device (911). For example, the wearable device (103) may identify the input to the external electronic device (911) within the FoV (907) and display the visual object (921) based on the input. The FoV (907) of the wearable device (103) may include a real environment (e.g., an AR environment) viewed through a camera of the wearable device (103) or an image rendered for the real environment (e.g., a VR environment).
[0193] According to one embodiment, the input may include a drag input (931) from an external electronic device (911) toward an external electronic device (912), or a drag input (932) from an external electronic device (911) toward an external electronic device (912). Referring to example (902), the wearable device (103) may identify a drag input (931). Alternatively, referring to example (903), the wearable device (103) may identify a drag input (932).
[0194] Referring to example (902), the wearable device (103) may transmit a request signal to the external electronic device (911) and the external electronic device (912) based on identifying the drag input (931). For example, the request signal based on the drag input (931) may be used to establish the second connection for the first function. For example, the first function may include a function of performing control on the external electronic device (911) based on a touch input to the external electronic device (912). However, the embodiments of the present disclosure are not limited thereto. For example, the first function may include other functions. Based on receiving a response signal indicating that the second connection for the first function is established, the wearable device (103) may display a visual object (922). For example, the visual object (922) may include information about the second connection for the first function. For example, information about the second connection for the first function may include information about the first function, whether data is being transmitted or received by the second connection, the time at which the second connection was established, the type of the second connection (e.g., BT, Wi-Fi), the communication status of the second connection (e.g., signal quality), or the subject of the second connection (e.g., information for distinguishing between a pairing device and a connection device in the case of BT).
[0195] Referring to example (903), the wearable device (103) may transmit a request signal to the external electronic device (911) and the external electronic device (912) based on identifying the drag input (932). For example, the request signal based on the drag input (932) may be used to establish the second connection for a second function different from the first function. For example, the second function may include a screen mirroring function for displaying a screen displayed on the external electronic device (912) on the external electronic device (911). However, the embodiments of the present disclosure are not limited thereto. For example, the second function may include other functions. Based on receiving a response signal indicating that the second connection for the second function is established, the wearable device (103) may display a visual object (923). For example, the visual object (923) may include information about the second connection for the second function. For example, information about the second connection for the second function may include information about the second function, whether data is being transmitted or received by the second connection, the time at which the second connection was established, the type of the second connection (e.g., BT, Wi-Fi), the communication status of the second connection (e.g., signal quality), or the subject of the second connection (e.g., information for distinguishing between a pairing device and a connection device in the case of BT).
[0196] In FIG. 9, the visual object (922) indicating the second connection for the first function and the visual object (923) indicating the second connection for the second function are illustrated as having the same shape, but the embodiment of the present disclosure is not limited thereto. For example, the visual object (922) indicating the second connection for the first function and the visual object (923) indicating the second connection for the second function may be formed with different shapes.
[0197] FIG. 10A illustrates an example of a method for displaying a visual object indicating a connection status of an external electronic device and displaying another visual object based on an input to the visual object.
[0198] The wearable device (103) of FIG. 10A may represent an example of the wearable device (103) of FIG. 5. FIG. 10A illustrates examples (1001, 1002, 1003, 1004) of displaying a visual object (1041) indicating a first connection between the wearable device (103) and an external electronic device (1011), and displaying a visual object (1042) indicating a second connection between the external electronic device (1011) and an external electronic device (1012).
[0199] Referring to example (1001), the wearable device (103) can display an external electronic device (1011). The wearable device (103) can display an external electronic device (1012). The wearable device (103) can display an icon (1020) for a function of managing a connection. For example, the wearable device (103) can display a screen including the external electronic device (1011), the external electronic device (1012), and the icon (1020) through the display (503). At this time, the wearable device (103) can be connected to the external electronic device (1011).
[0200] Referring to example (1002), the wearable device (103) may display a visual object (1041) representing the first connection based on identifying an input (1031) for an icon (1020). For example, the wearable device (103) may obtain the input (1031). For example, the input (1031) may include a user input to the screen. For example, the wearable device (103) may display a visual object (1041) in response to identifying the input (1031). While FIG. 10A illustrates an example of displaying a visual object (1041) based on an input (1031), embodiments of the present disclosure are not limited thereto. For example, the wearable device (103) may identify an event based on voice recognition and display a visual object (1041) based on the event. Also, for example, in example (1002), a visual object (1041) representing a connection is illustrated, but embodiments of the present disclosure are not limited thereto. For example, the visual object (1041) may include information about the connection. For example, the information about the connection may include whether data is being transmitted or received by the connection, the time the connection was established, the type of the connection (e.g., BT, Wi-Fi), the communication status of the connection (e.g., signal quality), or the subject of the connection (e.g., information for distinguishing between a paired device and a connected device in the case of BT). In addition, for example, the wearable device (103) may also display a visual object (1041) representing a connection and a visual object representing connection possibility and connections between other devices based on an input (1031) for an icon (1020).
[0201] Referring to example (1003), the wearable device (103) can identify an input (1032) for a visual object (1041). For example, the input (1032) can include a drag input for switching from the first connection to the second connection between the external electronic device (1011) and the external electronic device (1012). Based on identifying the drag input, the wearable device (103) can transmit a request signal for establishing the second connection to the external electronic device (1011) and the external electronic device (1012).
[0202] Referring to example (1004), the wearable device (103) may display a visual object (1042) indicating the second connection. For example, the wearable device (103) may receive a response signal to the request signal from an external electronic device (1011) or an external electronic device (1012). For example, the wearable device (103) may display the visual object (1042) based on an operation of receiving the response signal indicating that the second connection is established.
[0203] FIG. 10b illustrates an example of a method for displaying a visual object indicating the connection status of an external electronic device and disconnecting the connection based on input to the visual object.
[0204] The wearable device (103) of FIG. 10B may represent an example of the wearable device (103) of FIG. 5. FIG. 10B illustrates embodiments (1051, 1052) that display a visual object (1070) indicating a third connection between an external electronic device (1061) and an external electronic device (1062), and disconnect the third connection based on an input to the visual object.
[0205] Referring to embodiment (1051), the wearable device (103) may display an external electronic device (1061), an external electronic device (1062), and an external electronic device (1063). For example, the wearable device (103) may display a screen including the external electronic device (1061), the external electronic device (1062), and the external electronic device (1063) through the display (503). Referring to embodiment (1051), the wearable device (103) may display a visual object (1070) indicating the third connection between the external electronic device (1061) and the external electronic device (1062). For example, the wearable device (103) may display the visual object (1070) based on identifying an event related to the third connection. For example, the event may include identification of an input to an external electronic device (1061) or an external electronic device (1062), or an input for a change in the third connection.
[0206] According to one embodiment, the wearable device (103) may identify an input (1080) for a visual object (1070). For example, the input (1080) may include a gesture for disconnecting the third connection indicated by the visual object (1070). For example, the input (1080) may include the gesture for disconnecting a line portion of the visual object (1070). Based on identifying the input (1080), the wearable device (103) may transmit a request signal for disconnecting the third connection to the external electronic device (1061) and the external electronic device (1062).
[0207] Referring to embodiment (1052), the wearable device (103) may receive a response signal to the request signal. For example, the wearable device (103) may stop displaying the visual object (1070) based on receiving the response signal indicating the disconnection of the third connection.
[0208] Additionally, according to a specific connection policy, the external electronic device (1061) may connect with the external electronic device (1063) in response to the third connection being released. For example, a fourth connection may be established between the external electronic device (1061) and the external electronic device (1063). The wearable device (103) may identify that the fourth connection is established. For example, the wearable device (103) may identify that the fourth connection is established based on information obtained from the external electronic device (1061) connected to the wearable device (103) or from another electronic device (e.g., the external electronic device (540) or the server (530) of FIG. 5) connected to the external electronic device (1061). The wearable device (103) may display a visual object (1090) indicating the fourth connection.
[0209] Figure 11 illustrates examples of visual objects that indicate connection status.
[0210] Figure 11 illustrates embodiments (1101, 1102, 1103) of visual objects representing the connection status. For example, the connection status may include a connection between electronic devices and the possibility of connection between electronic devices.
[0211] Referring to embodiment (1101), a visual object (1130) may be used to indicate a connection between a first electronic device (1110) and a second electronic device (1120). For example, the visual object (1130) may include a first icon (1131) representing a connection to the first electronic device (1110), a second icon (1132) representing a connection to the second electronic device (1120), and / or a line (1133) representing a connection between the first icon (1131) and the second icon (1132). For example, each of the first icon (1131) and the second icon (1132) may include an arrow. Alternatively, each of the first icon (1131) and the second icon (1132) may have a concentric plug (or socket, electrical outlet) shape.
[0212] Referring to embodiment (1102), a visual object (1140) may be used to indicate a connection between a first electronic device (1110) and a second electronic device (1120). For example, the visual object (1140) may include a first icon (1141) representing a connection to the first electronic device (1110), a second icon (1142) representing a connection to the second electronic device (1120), and a line (1143) representing a connection between the first icon (1141) and the second icon (1142). For example, each of the first icon (1141) and the second icon (1142) may include a plurality of arrows. However, the embodiment of the present disclosure is not limited thereto. The first icon (1141) and the second icon (1142) may correspond to the first icon (1131) and the second icon (1132). For example, line (1143) may be displayed upon identifying an input to a visual object (1140). For example, line (1143) may not be displayed when a visual object (1140) is displayed to indicate a connection between a first electronic device (1110) and a second electronic device (1120). Thereafter, when an input to the visual object (1140) is identified, line (1143) may be displayed.
[0213] Referring to embodiment (1103), a visual object (1150) may be used to indicate a connection between a first electronic device (1110) and a second electronic device (1120). For example, the visual object (1150) may include a first icon (1151) indicating a connection to the first electronic device (1110) and a second icon (1152) indicating a connection to the second electronic device (1120). For example, each of the first icon (1151) and the second icon (1152) may include an index. For example, the index may be used to map the order of the connection between the first electronic device (1110) and the second electronic device (1120) and the electronic devices of the connection. In one embodiment, the first icon and the second icon displayed in relation to other electronic devices for other connections may include an index (e.g., 2) that is different from the index (e.g., 1) included in the first icon (1151) and the second icon (1152).
[0214] FIG. 11 illustrates visual objects (1130, 1140, 1150) including icons and lines, but the present disclosure is not limited thereto. For example, each of the visual objects (1130, 1140, 1150) may include only lines.
[0215] In FIG. 11, visual objects (1130, 1140, 1150) representing a connection are illustrated, but embodiments of the present disclosure are not limited thereto. For example, the visual objects (1130, 1140, 1150) may include information about the connection. For example, the information about the connection may include whether data is being transmitted or received through the connection, the time the connection was established, the type of the connection (e.g., BT, Wi-Fi), the communication status of the connection (e.g., signal quality), or the subject of the connection (e.g., information for distinguishing between a pairing device and a connection device in the case of BT).
[0216] FIG. 12 illustrates an example of a method for displaying a visual object indicating a connection status between a wearable device and an external electronic device.
[0217] At least some of the methods of FIG. 12 may be performed by the wearable device (103) of FIG. 5. For example, at least some of the methods may be controlled by the processor (501) of the wearable device (103). In the embodiments described below, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.
[0218] In operation (1200), the wearable device (103) may detect (or identify) one or more external electronic devices using the communication circuit (509). For example, the one or more external electronic devices may include a first external electronic device.
[0219] In operation (1210), the wearable device (103) may establish a first connection with the first external electronic device. For example, the wearable device (103) may identify the first external electronic device among the one or more external electronic devices. The wearable device (103) may establish the first connection with the first external electronic device.
[0220] According to one embodiment, the wearable device (103) can identify one or more external electronic devices. For example, the wearable device (103) can identify the one or more external electronic devices based on the communication circuit (509). The one or more external electronic devices can include the first external electronic device. For example, the wearable device (103) can identify the one or more external electronic devices using short-range communication using the communication circuit (509).
[0221] According to one embodiment, the one or more external electronic devices may include at least one of an electronic device connectable via short-range wireless communication using the communication circuit (509), an electronic device having the same account information as the account information of the wearable device (103) identified using the communication circuit (509), an electronic device within a specified distance from the wearable device (103) identified using the communication circuit (509), or an electronic device connected to the same access point (AP) as the wearable device (103) identified using the communication circuit (509).
[0222] According to one embodiment, the wearable device (103) may identify a connection status of the one or more external electronic devices. For example, the wearable device (103) may transmit a first signal to each of the one or more external electronic devices. For example, the wearable device (103) may broadcast or multicast the first signal to each of the one or more external electronic devices. According to one embodiment, the wearable device (103) may receive a second signal from at least some of the one or more external electronic devices in response to the first signal. For example, each of the first signal and the second signal may be referred to as an advertising signal, an identification signal, a broadcasting signal, or a multicasting signal. According to one embodiment, the wearable device (103) may establish a connection with at least some of the external electronic devices or identify the connection status of at least some of the external electronic devices based on the second signal. For example, the wearable device (103) may transmit the first signal at a specified interval or based on at least one of a user's input.
[0223] According to one embodiment, the first signal may include at least one of identification information of the wearable device (103), account information of the user of the wearable device (103), connection status information of the wearable device (103), connection record information of the wearable device (103), transmission intensity information of the first signal, detection area information based on the first signal, battery information of the wearable device (103), or audio output information of the wearable device (103). In addition, according to one embodiment, the second signal may include at least one of identification information of the first external electronic device, account information of the user of the first external electronic device, connection status information of the first external electronic device, connection record information of the first external electronic device, transmission intensity information of the second signal, detection area information based on the second signal, battery information of the first external electronic device, or audio output information of the first external electronic device. The first external electronic device may be included in at least a portion of the one or more external electronic devices.
[0224] In one embodiment, the wearable device (103) may identify the first connection. For example, the first connection may include a connection between the wearable device (103) and a first external electronic device among the one or more external electronic devices. In one example, the wearable device (103) may identify the first connection based on the first signal and the second signal.
[0225] In operation (1220), the wearable device (103) can obtain an image of the environment of the wearable device (103) through the camera (507). For example, the wearable device (103) can obtain an image of the actual environment around the wearable device (103) or an image of a virtual environment. The image can be related to the FoV of the wearable device (103).
[0226] In operation (1230), the wearable device (103) may identify a second external electronic device within the image. For example, the wearable device (103) may identify the second external electronic device within the FoV among the one or more external electronic devices. The direction of the FoV may change depending on the gaze direction of the user wearing the wearable device (103).
[0227] In operation (1240), the wearable device (103) may display a first visual object representing the first connection based on identifying an event related to the first connection. For example, the wearable device (103) may display a first visual object representing the first connection through the display (503) based on identifying an event related to the first connection.
[0228] According to one embodiment, the wearable device (103) may obtain the event from another external electronic device. For example, the other external electronic device may include the server (530) or the external electronic device (540) of FIG. 5. Alternatively, for example, the wearable device (103) may obtain the event from an external electronic device that has established the first connection with the wearable device (103), or the wearable device (103) may identify the event for the first connection. For example, the wearable device (103) may generate (or render) the first visual object and display the generated first visual object based on identifying the event related to the first connection.
[0229] For example, an event related to the first connection may include at least one of an input for a visual object for establishing a connection displayed through a display (503), receiving information for confirming whether a connection to an external electronic device (e.g., external electronic devices (520) of FIG. 5) or a wearable device (103) is possible from an external electronic device (e.g., a server (530) or an external electronic device (540) of FIG. 5) connected to the wearable device (103), executing a function for performing multiple connections, or an input for changing a connection.
[0230] In one embodiment, the wearable device (103) may display the first visual object based on identifying (or receiving) the event. For example, the first visual object may include a visual object for indicating the first connection. For example, the wearable device (103) may display the first visual object for a specified length of time from the time when the event is identified. Alternatively, for example, the wearable device (103) may display the first visual object when the user's movement of the wearable device (103) is within a specified range.
[0231] According to one embodiment, the first visual object may be displayed with respect to the electronic devices that have established the first connection. For example, the wearable device (103) may display the first visual object between the wearable device (103) and the first external electronic device. For example, when the first external electronic device is positioned within the FoV, the first visual object may extend from the first external electronic device to the wearable device (103). Alternatively, when the first external electronic device is positioned outside the FoV, the first visual object may extend from an area corresponding to the position of the first external electronic device to the wearable device (103).
[0232] In operation (1250), the wearable device (103) may obtain input regarding the first visual object. For example, the input may include an interaction performed by a user of the wearable device (103) regarding the first visual object.
[0233] In operation (1260), the wearable device (103) may, based on the input, cease displaying the first visual object and display a second visual object indicating a second connection. For example, the input obtained for the first visual object may include an input for establishing the second connection between the first external electronic device and the second external electronic device. For example, the input obtained for the first visual object may include a drag (or drag and drop) input.
[0234] According to one embodiment, the wearable device (103) may transmit a request signal for establishing the second connection. For example, the wearable device (103) may transmit the request signal to the first external electronic device and the second external electronic device. In this case, the request signal transmitted to the first external electronic device may be used to request that the first connection be released (or disconnected) and the second connection be established. The request signal transmitted to the second external electronic device may be used to request that the second connection be established.
[0235] According to one embodiment, when the user's input is the drag input, the second connection may be established for executing a designated function based on the direction of the drag input. For example, when the direction of the drag input is a first direction from the first external electronic device to the second external electronic device, the second connection may be established for executing a first function. Conversely, when the direction of the drag input is a second direction from the second external electronic device to the first external electronic device, the second connection may be established for executing a second function different from the first function. In this case, the wearable device (103) may transmit the request signal for establishing the second connection for executing the first function or the second function.
[0236] According to one embodiment, the wearable device (103) may receive a response signal. For example, the wearable device (103) may receive the response signal to the request signal. In the example, the response signal may indicate that the second connection is established. For example, the response signal may be received from the first external electronic device or the second external electronic device. The wearable device (103) may identify that the second connection is established based on receiving the response signal.
[0237] According to one embodiment, the wearable device (103) may, based on receiving the response signal, stop displaying the first visual object and display the second visual object. For example, the wearable device (103) may stop displaying the first visual object first and then display the second visual object, or may display the second visual object first and then stop displaying the first visual object. For example, the description of the second visual object may be substantially identical to the description of the first visual object.
[0238] Referring to the above, an example is described in which the wearable device (103) displays the first visual object and the second visual object indicating a connection, but the embodiments of the present disclosure are not limited thereto. For example, each of the first visual object and the second visual object may include information about the connection. For example, the information about the connection may include whether data is being transmitted and received by the connection, the time at which the connection was established, the type of the connection (e.g., BT, Wi-Fi), the communication status of the connection (e.g., signal quality), or the subject of the connection (e.g., information for distinguishing between a pairing device and a connection device in the case of BT).
[0239] FIG. 13 illustrates an example of a method for displaying a visual object indicating a connection status between external electronic devices.
[0240] At least some of the methods of FIG. 13 may be performed by the wearable device (103) of FIG. 5. For example, at least some of the methods may be controlled by the processor (501) of the wearable device (103). In the embodiments described below, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.
[0241] In operation (1300), the wearable device (103) may detect (or identify) one or more external electronic devices using the communication circuit (509). For example, the one or more external electronic devices may include a first external electronic device and a second external electronic device.
[0242] In operation (1310), the wearable device (103) can obtain an image of the environment of the wearable device (103) through the camera (507). For example, the wearable device (103) can obtain an image of the actual environment around the wearable device (103) or an image of a virtual environment. The image can be related to the FoV of the wearable device (103).
[0243] In operation (1320), the wearable device (103) may identify the first external electronic device and the second external electronic device within the image. For example, the wearable device (103) may identify the first external electronic device and the second external electronic device among the one or more external electronic devices. A first connection may be established between the first external electronic device and the second external electronic device.
[0244] According to one embodiment, the wearable device (103) can identify one or more external electronic devices. For example, the wearable device (103) can identify one or more external electronic devices using the communication circuit (509). For example, the wearable device (103) can identify one or more external electronic devices through short-range communication using the communication circuit (509).
[0245] According to one embodiment, the one or more external electronic devices may include at least one of an electronic device connectable via short-range wireless communication using the communication circuit (509), an electronic device having the same account information as the account information of the wearable device (103) identified using the communication circuit (509), an electronic device within a specified distance from the wearable device (103) identified using the communication circuit (509), or an electronic device connected to the same access point (AP) as the wearable device (103) identified using the communication circuit (509).
[0246] According to one embodiment, the wearable device (103) may identify a connection status of the one or more external electronic devices. For example, the wearable device (103) may transmit a first signal to each of the one or more external electronic devices. For example, the wearable device (103) may broadcast or multicast the first signal to each of the one or more external electronic devices. According to one embodiment, the wearable device (103) may receive a second signal from at least some of the one or more external electronic devices in response to the first signal. For example, each of the first signal and the second signal may be referred to as an advertising signal, an identification signal, a broadcasting signal, or a multicasting signal. According to one embodiment, the wearable device (103) may establish a connection with at least some of the external electronic devices or identify the connection status of at least some of the external electronic devices based on the second signal. For example, the wearable device (103) may transmit the first signal at a specified interval or based on at least one of a user's input.
[0247] According to one embodiment, the first signal may include at least one of identification information of the wearable device (103), account information of a user of the wearable device (103), connection status information of the wearable device (103), connection record information of the wearable device (103), transmission intensity information of the first signal, detection area information based on the first signal, battery information of the wearable device (103), or sound output information of the wearable device (103). In addition, according to one embodiment, the second signal may include at least one of identification information of an external electronic device (e.g., the first external electronic device or the second external electronic device), account information of a user of the external electronic device (e.g., the first external electronic device or the second external electronic device), connection status information of the external electronic device (e.g., the first external electronic device or the second external electronic device), connection record information of the external electronic device (e.g., the first external electronic device or the second external electronic device), transmission intensity information of the second signal, detection area information based on the second signal, battery information of the external electronic device (e.g., the first external electronic device or the second external electronic device), or sound output information of the external electronic device (e.g., the first external electronic device or the second external electronic device).
[0248] In one embodiment, the wearable device (103) may identify the first connection. For example, the first connection may include a connection between the first external electronic device and the second external electronic device among the one or more external electronic devices. In one example, the wearable device (103) may identify the first connection based on the first signal and the second signal.
[0249] In operation (1330), the wearable device (103) may display a first visual object representing the first connection based on identifying an event related to the first connection. For example, the wearable device (103) may display a first visual object representing the first connection through the display (503) based on identifying an event related to the first connection.
[0250] According to one embodiment, the wearable device (103) may obtain the event from another external electronic device. For example, the other external electronic device may include the server (530) or the external electronic device (540) of FIG. 5. Alternatively, for example, the wearable device (103) may obtain the event from an external electronic device that has established the first connection with the wearable device (103), or the wearable device (103) may identify the event for the first connection. For example, the wearable device (103) may generate (or render) the first visual object and display the generated first visual object based on identifying the event related to the first connection.
[0251] For example, an event related to the first connection may include at least one of an input for a visual object for establishing a connection displayed through a display (503), receiving information for confirming whether a connection to an external electronic device (e.g., external electronic devices (520) of FIG. 5) or a wearable device (103) is possible from an external electronic device (e.g., a server (530) or an external electronic device (540) of FIG. 5) connected to the wearable device (103), executing a function for performing multiple connections, or an input for changing a connection.
[0252] In one embodiment, the wearable device (103) may display the first visual object based on identifying (or receiving) the event. For example, the first visual object may include a visual object for indicating the first connection. For example, the wearable device (103) may display the first visual object for a specified length of time from the time when the event is identified. Alternatively, for example, the wearable device (103) may display the first visual object when the user's movement of the wearable device (103) is within a specified range.
[0253] According to one embodiment, the first visual object may be displayed with respect to the electronic devices that have established the first connection. For example, the wearable device (103) may display the first visual object between the first external electronic device and the second external electronic device. For example, when the first external electronic device is positioned within the FoV, the first visual object may extend from the first external electronic device to the second external electronic device. Alternatively, when the first external electronic device is positioned outside the FoV, the first visual object may extend from an area corresponding to a location of the first external electronic device to the second external electronic device.
[0254] In operation (1340), the wearable device (103) may obtain input regarding the first visual object. For example, the input may include an interaction performed by a user of the wearable device (103) regarding the first visual object.
[0255] In operation (1350), the wearable device (103) may, based on the input, cease displaying the first visual object and display a second visual object indicating a second connection. For example, the input obtained for the first visual object may include an input for establishing the second connection between the second external electronic device and a third external electronic device. For example, the input obtained for the first visual object may include a drag (or drag and drop) input. The third external electronic device may be included in the one or more external electronic devices.
[0256] According to one embodiment, the wearable device (103) may transmit a request signal for establishing the second connection. For example, the wearable device (103) may transmit the request signal to the first external electronic device, the second external electronic device, and the third external electronic device. At this time, the request signal transmitted to the first external electronic device may be used to request to release (or disconnect) the first connection. The request signal transmitted to the second external electronic device may be used to request to release the first connection and establish the second connection. The request signal transmitted to the third external electronic device may be used to request to establish the second connection.
[0257] According to one embodiment, when the user's input is the drag input, the second connection may be established for executing a designated function based on the direction of the drag input. For example, when the direction of the drag input is a first direction from the second external electronic device to the third external electronic device, the second connection may be established for executing a first function. Conversely, when the direction of the drag input is a second direction from the third external electronic device to the second external electronic device, the second connection may be established for executing a second function different from the first function. In this case, the wearable device (103) may transmit the request signal for establishing the second connection for executing the first function or the second function.
[0258] According to one embodiment, the wearable device (103) may receive a response signal. For example, the wearable device (103) may receive the response signal to the request signal. In the example, the response signal may indicate that the second connection is established. For example, the response signal may be received from the second external electronic device or the third external electronic device. The wearable device (103) may identify that the second connection is established based on receiving the response signal.
[0259] According to one embodiment, the wearable device (103) may, based on receiving the response signal, stop displaying the first visual object and display the second visual object. For example, the wearable device (103) may stop displaying the first visual object first and then display the second visual object, or may display the second visual object first and then stop displaying the first visual object. For example, the description of the second visual object may be substantially identical to the description of the first visual object.
[0260] Referring to the above, an example is described in which the wearable device (103) displays the first visual object and the second visual object indicating a connection, but the embodiments of the present disclosure are not limited thereto. For example, each of the first visual object and the second visual object may include information about the connection. For example, the information about the connection may include whether data is being transmitted and received by the connection, the time at which the connection was established, the type of the connection (e.g., BT, Wi-Fi), the communication status of the connection (e.g., signal quality), or the subject of the connection (e.g., information for distinguishing between a pairing device and a connection device in the case of BT).
[0261] As described above, the wearable device (103) may include a display (503). The wearable device (103) may include a communication circuit (509). The wearable device (103) may include a camera (507). The wearable device (103) may include at least one processor (501) including a processing circuit. The wearable device (103) may include one or more storage media and may include a memory for storing instructions. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to detect a plurality of external electronic devices using the communication circuit (509). The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to establish a first connection with a first external electronic device among the plurality of external electronic devices using the communication circuit (509). The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to acquire an image of the surroundings of the wearable device (103) through the camera (507). The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to identify a second external electronic device within the image among the plurality of external electronic devices.The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to display, through the display (503), a first visual object representing the first connection based on identifying an event associated with the first connection. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to obtain an input for the first visual object to establish a second connection between the first external electronic device and the second external electronic device. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to, based on the input, cease displaying the first visual object and display, through the display (503), a second visual object representing the second connection.
[0262] In one embodiment, the instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to transmit a first signal to each of the plurality of external electronic devices. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to receive a second signal from the first external electronic device in response to the first signal. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to establish the first connection based on the second signal.
[0263] According to one embodiment, the first signal may include at least one of identification information of the wearable device (103), account information of a user of the wearable device (103), connection status information of the wearable device (103), connection record information of the wearable device (103), or transmission intensity information of the first signal. The second signal may include at least one of identification information of the first external electronic device, account information of a user of the first external electronic device, connection status information of the first external electronic device, connection record information of the first external electronic device, or transmission intensity information of the second signal.
[0264] According to one embodiment, the plurality of external electronic devices may include at least one of an electronic device having the same account information as the account information of the wearable device (103) identified using the communication circuit (509), or an electronic device within a specified distance from the wearable device (103) identified using the communication circuit (509).
[0265] According to one embodiment, the event related to the first connection may include at least one of: an input for a visual object for establishing the first connection displayed through the display (503), receiving information for confirming whether a connection to the first external electronic device or the wearable device (103) is possible from a third external electronic device connected to the wearable device (103), executing a function for performing multiple connections, or an input for changing the first connection.
[0266] According to one embodiment, the first visual object may be displayed within a specified length of time from the timing at which the event is identified or while the movement of the user of the wearable device (103) is within a specified range.
[0267] According to one embodiment, the first visual object may include a line representing a connection between the first external electronic device and the wearable device (103).
[0268] According to one embodiment, when the first external electronic device is positioned within the FoV, the first visual object may extend from the first external electronic device to the wearable device (103). When the first external electronic device is positioned outside the FoV, the first visual object may extend from an area corresponding to a position of the first external electronic device within the FoV to the wearable device (103).
[0269] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to transmit a capability information inquiry message to the first external electronic device requesting information about connectable electronic devices. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to receive, in response to the capability information inquiry message, a capability information message from the first external electronic device indicating one or more connectable external electronic devices, including the second external electronic device.
[0270] In one embodiment, the instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to identify the event associated with the first connection. The instructions, when individually or collectively executed by the at least one processor (501), may further cause the wearable device (103) to, in response to the event, display an indicator indicating that a connection is possible between the first external electronic device and the second external electronic device, identified based on the capability information message, together with the first visual object representing the first connection.
[0271] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to, in response to obtaining the input, disconnect the first connection and transmit a request signal for establishing the second connection to the first external electronic device, and transmit another request signal for establishing the second connection to the second external electronic device. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to receive a response signal from the first external electronic device or the second external electronic device, indicating that the second connection has been established. The above instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to, in response to receiving the response signal, stop displaying the first visual object and display the second visual object representing the second connection.
[0272] In one embodiment, the request signal may be used to establish the second connection for executing a first function when the input is a first input from the first external electronic device to the second external electronic device. The request signal may be used to establish the second connection for executing a second function different from the first function when the input is a second input from the second external electronic device to the first external electronic device.
[0273] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (501), may further cause the wearable device (103) to display, through the display (503), a third visual object indicating a third connection between a third external electronic device connected to the first external electronic device and the first external electronic device, based on identifying the event associated with the first connection.
[0274] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to transmit a request signal for disconnection of the third connection to each of the first external electronic device and the third external electronic device based on another input obtained with respect to the third visual object for disconnection of the third connection. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to receive a response signal from the first external electronic device or the third external electronic device, indicating that the third connection has been disconnected. The above instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to stop displaying the third visual object in response to receiving the response signal.
[0275] As described above, the wearable device (103) may include a display (503). The wearable device (103) may include a communication circuit (509). The wearable device (103) may include a camera (507). The wearable device (103) may include at least one processor (501). The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to detect a plurality of external electronic devices using the communication circuit (509). The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to acquire an image of the surroundings of the wearable device (103) using the camera (507). The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to identify a first external electronic device and a second external electronic device in the image from among the plurality of external electronic devices using the communication circuit (509). The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to display, through the display (503), a first visual object representing the first connection based on identifying an event associated with the first connection between the first external electronic device and the second external electronic device.The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to obtain an input for the first visual object for establishing a second connection between the second external electronic device and a third external electronic device among the plurality of external electronic devices. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to, based on the input, cease displaying the first visual object and display, through the display (503), a second visual object representing the second connection.
[0276] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to transmit a first signal to each of the plurality of external electronic devices. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to receive a second signal from the first external electronic device in response to the first signal. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to receive a third signal from the second external electronic device in response to the first signal. The above instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to identify the first connection based on the second signal or the third signal.
[0277] According to one embodiment, the plurality of external electronic devices may include at least one of an electronic device having the same account information as the account information of the wearable device (103) identified using the communication circuit (509), or an electronic device within a specified distance from the wearable device (103) identified using the communication circuit (509).
[0278] According to one embodiment, the event related to the first connection may include at least one of: an input for a visual object for establishing the first connection displayed through the display (503), receiving information for confirming whether a connection to the first external electronic device or the second external electronic device is possible from a fourth external electronic device connected to the wearable device (103), executing a function for performing multiple connections, or an input for changing the first connection.
[0279] According to one embodiment, the first visual object may include a line representing a connection between the first external electronic device and the second external electronic device.
[0280] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to, in response to obtaining the input, disconnect the first connection and transmit a request signal for establishing the second connection to each of the first external electronic device and the second external electronic device. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to receive a response signal from the second external electronic device or the third external electronic device, indicating that the second connection has been established. The instructions, when individually or collectively executed by the at least one processor (501), may cause the wearable device (103) to, in response to receiving the response signal, stop displaying the first visual object and display the second visual object indicating the second connection.
[0281] Electronic devices according to one or more embodiments disclosed herein may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to embodiments disclosed herein are not limited to the aforementioned devices.
[0282] It should be understood that one or more embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but rather to encompass various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said item, unless the context clearly dictates otherwise. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish the corresponding component from other corresponding components and do not limit the corresponding components in any other respect (e.g., importance or order). When a (e.g., first) component is referred to as "coupled" or "connected" to another (e.g., second) component, with or without the terms "functionally" or "communicatively," it means that said component can be connected to said other component directly (e.g., wired), wirelessly, or through a third component.
[0283] The term "module" as used in one or more embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or portion of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).
[0284] One or more embodiments may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.
[0285] According to one embodiment, a method according to one or more embodiments disclosed in the present document may be provided as a computer program product. The computer program product may be traded between sellers and buyers as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or may be provided through an application store (e.g., Play Store). TM) or directly between two user devices (e.g., smart phones), online distribution (e.g., downloading or uploading). In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.
[0286] According to one or more embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and arranged in other components. According to one or more embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, according to one or more embodiments of the present disclosure, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to one or more embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.
Claims
1. In a wearable device (103), display (503); Camera (507); Communication circuit (509); At least one processor (501) comprising a processing circuit; and A memory (511) storing instructions and including one or more storage media, The above instructions, when individually or collectively executed by the at least one processor (501), cause the wearable device (103) to: Identifying multiple external electronic devices using the above communication circuit (509); Establishing a first connection with a first external electronic device among the plurality of external electronic devices using the above communication circuit (509); Obtain an image of the environment of the wearable device (103) through the camera (507); Identifying a second external electronic device among the plurality of external electronic devices in the acquired image; Based on identifying an event related to the first connection, displaying a first visual object representing the first connection through the display (503); Obtaining input for said first visual object to establish a second connection between said first external electronic device and said second external electronic device; and Based on the input, causing the display of the first visual object to be stopped and the second visual object representing the second connection to be displayed through the display (503). Wearable device (103).
2. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (501), cause the wearable device (103) to: Transmitting a first signal to each of said plurality of external electronic devices; In response to the first signal, receiving a second signal from the first external electronic device; and further causing, based on the second signal, to establish the first connection; Wearable device (103).
3. In claim 2, The first signal includes at least one of identification information of the wearable device (103), account information of the user of the wearable device (103), connection status information of the wearable device (103), connection record information of the wearable device (103), or transmission strength information of the first signal. The second signal includes at least one of identification information of the first external electronic device, account information of a user of the first external electronic device, connection status information of the first external electronic device, connection record information of the first external electronic device, or transmission strength information of the second signal. Wearable device (103).
4. In claim 1, The above multiple external electronic devices are: An electronic device having the same account information as the account information of the wearable device (103) identified using the above communication circuit (509), or At least one of the electronic devices, which is within a specified distance from the wearable device (103) identified using the communication circuit (509), Wearable device (103).
5. In claim 1, The above events related to the first connection are: Input for a visual object for establishing the first connection displayed through the above display (503); Information for checking whether the first external electronic device or the wearable device (103) can be connected is received from a third external electronic device connected to the wearable device (103). Execution of a function to perform multiple connections, or At least one of the inputs for changing the first connection is included, Wearable device (103).
6. In claim 1, The first visual object is displayed within a specified length of time from the timing at which the event is identified or while the movement of the user of the wearable device (103) is within a specified range, and The first visual object comprises a line representing the first connection between the first external electronic device and the wearable device (103). Wearable device (103).
7. In claim 1, When the first external electronic device is positioned within the FoV, the first visual object extends from the first external electronic device to the wearable device (103), When the first external electronic device is located outside the FoV, the first visual object extends from an area corresponding to the location of the first external electronic device within the FoV to the wearable device (103). Wearable device (103).
8. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (501), cause the wearable device (103) to: Transmitting a capability information inquiry message to the first external electronic device, requesting information about a connectable electronic device; and In response to the capability information inquiry message, further causing the first external electronic device including the second external electronic device to receive, from the first external electronic device, a capability information message indicating one or more external electronic devices connectable to the first external electronic device. Wearable device (103).
9. In claim 8, The above instructions, when individually or collectively executed by the at least one processor (501), cause the wearable device (103) to: identifying said event associated with said first connection; and further causing an indicator to be displayed, together with the first visual object representing the first connection, based on the identified event associated with the first connection; The above indicator indicates that the second connection between the first external electronic device and the second external electronic device is possible based on the capability information message. Wearable device (103).
10. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (501), cause the wearable device (103) to: In response to obtaining said input, disconnecting said first connection and transmitting a request signal for establishing said second connection to said first external electronic device, and transmitting another request signal for establishing said second connection to said second external electronic device; Receiving a response signal indicating that the second connection is established, from the first external electronic device or the second external electronic device; and Further causing, based on receiving said response signal, to stop displaying said first visual object and display said second visual object representing said second connection. Wearable device (103).
11. In claim 10, The above request signal is used to establish the second connection for executing the first function, when the input is a first input from the first external electronic device to the second external electronic device, and The above request signal is used to establish the second connection for executing a second function different from the first function, when the input is a second input from the second external electronic device to the first external electronic device. Wearable device (103).
12. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (501), cause the wearable device (103) to: Further causing a third visual object to be displayed, through the display (503), indicating a third connection between a third external electronic device connected to the first external electronic device and the first external electronic device, based on identifying the event associated with the first connection. Wearable device (103).
13. In claim 12, The above instructions, when individually or collectively executed by the at least one processor (501), cause the wearable device (103) to: Based on another input obtained with respect to the third visual object for disconnection of the third connection, transmitting a request signal for disconnection of the third connection to each of the first external electronic device and the third external electronic device; Receiving a response signal notifying that the third connection is disconnected from the first external electronic device or the third external electronic device; and further causing the display of the third visual object to be stopped based on receiving the above response signal; Wearable device (103).
14. In a method performed by a wearable device (103), An act of identifying multiple external electronic devices; An operation of establishing a first connection with a first external electronic device among the plurality of external electronic devices; An operation of acquiring an image of the environment of the above wearable device (103); An operation of identifying a second external electronic device in the acquired image among the plurality of external electronic devices; An action of displaying a first visual object representing the first connection based on identifying an event associated with the first connection; An operation of obtaining an input for the first visual object to establish a second connection between the first external electronic device and the second external electronic device; and Based on said input, comprising an action of ceasing display of said first visual object and displaying a second visual object representing said second connection; method.
15. A non-transitory computer-readable storage medium, when individually or collectively executed by at least one processor (501) of a wearable device (103) including a display (503), a camera (507), and a communication circuit (509): Identifying multiple external electronic devices using the above communication circuit (509); Establishing a first connection with a first external electronic device among the plurality of external electronic devices using the above communication circuit (509); Obtain an image of the environment of the wearable device (103) through the camera (507); Identifying a second external electronic device among the plurality of external electronic devices in the acquired image; Based on identifying an event related to the first connection, displaying a first visual object representing the first connection through the display (503); and Obtaining input for said first visual object to establish a second connection between said first external electronic device and said second external electronic device; storing one or more programs including instructions that cause, based on said input, to cease displaying said first visual object and to display, through said display (503), a second visual object representing said second connection; A non-transitory computer-readable storage medium.
Citation Information
Patent Citations
Method for generating content to be displayed at virtual area via augmented reality platform and electronic device supporting the same
KR1020170125618A
Electrode rolling apparatus comprising non-coating portion pressing part and electrode rolling method using the same
KR1020210144585A
An umbrella storage box
KR1020230166735A
Electrode assembly, battery, and battery pack and vehicle including the same
KR1020240011477A
Method and apparatus for monitoring control information in wireless communication system
KR1020240150077A