Wearable electronic device and control method for same wearable electronic device

Abstract

A wearable electronic device according to various embodiments of the present invention comprises: a display; at least one camera; a communication circuit; a processor; and a memory for storing instructions, wherein the instructions, when executed by the processor, cause the wearable electronic device to: perform pairing with an external electronic device through the communication circuit; recognize the external electronic device viewed through the at least one camera; transmit, to the external electronic device, information related to gaze coordinates and / or gesture coordinates of a user of the wearable electronic device; receive, from the external electronic device, a first execution screen played in the external electronic device through mirroring on the basis of the information related to the gaze coordinates and / or the gesture coordinates; and display, through the display, an expanded screen of the first execution screen received from the external electronic device, wherein the expanded screen of the first execution screen may be configured to be controllable through a gaze and / or a gesture of the user of the wearable electronic device, and the external electronic device may be configured to be controllable through a paired peripheral device.

Description

Wearable electronic device and method for controlling the wearable electronic device

[0001] Various embodiments of the present invention disclose a wearable electronic device and a method for controlling the wearable electronic device.

[0002] The use of wearable electronic devices is increasing, and various functions are being provided to such wearable electronic devices.

[0003] For example, the wearable electronic device may include various types such as virtual reality (AR) glasses, video see through (VST) mixed reality (MR) glasses, or head mounted displays (HMD).

[0004] The above-mentioned wearable electronic device can provide various virtual reality experiences through a display module. Users can experience various virtual reality experiences through the wearable electronic device.

[0005] The above information may be provided as background information to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art in connection with the present disclosure.

[0006] Wearable electronic devices can visually present content (e.g., application execution screens) provided by external electronic devices in a virtual space. Users can interact with the content provided in the virtual space to experience a variety of experiences.

[0007] Content displayed through the display module (e.g., display) of the wearable electronic device may not be controllable through the user's gaze and / or gestures.

[0008] When a user wears a wearable electronic device, the wearable electronic device may not automatically connect, reconnect, and / or disconnect with external electronic devices within a certain communicable range.

[0009] Various embodiments of the present invention can provide a wearable electronic device and method that can control a screen (e.g., a content execution screen) provided through a display (e.g., a display module) through a user's gaze and / or gesture.

[0010] Various embodiments of the present invention may provide a method by which, when a user wears a wearable electronic device, the wearable electronic device can automatically connect, reconnect, and / or disconnect from an external electronic device and / or peripheral device within a certain range.

[0011] The technical problems to be achieved in the present disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention belongs from the description below.

[0012] According to one embodiment of the present invention, a wearable electronic device may include a display, at least one camera, a communication circuit, at least one processor, and a memory storing instructions. According to one embodiment, the instructions, when executed by the at least one processor, may cause the wearable electronic device to perform pairing with an external electronic device through the communication circuit, recognize the external electronic device viewed through the at least one camera, transmit information related to gaze coordinates and / or gesture coordinates of a user of the wearable electronic device to the external electronic device, receive a first execution screen being played on the external electronic device from the external electronic device through mirroring based on the information related to the gaze coordinates and / or gesture coordinates, and display an expanded screen of the first execution screen received from the external electronic device through the display. According to one embodiment, the expanded screen of the first execution screen may be configured to be controllable through a gaze and / or gesture of a user of the wearable electronic device. In one embodiment, the external electronic device may be configured to be controllable via a paired peripheral device.

[0013] A method for controlling a wearable electronic device according to one embodiment of the present invention may include an operation in which the wearable electronic device performs pairing with an external electronic device through a communication circuit. According to one embodiment, the method may include an operation in which the wearable electronic device recognizes the external electronic device. According to one embodiment, the method may include an operation in which information related to gaze coordinates and / or gesture coordinates of a user of the wearable electronic device is transmitted to the external electronic device. According to one embodiment, the method may include an operation in which a first execution screen being played on the external electronic device is received from the external electronic device through mirroring based on information related to the gaze coordinates and / or gesture coordinates. According to one embodiment, the method may include an operation in which an expanded screen of the first execution screen received from the external electronic device is displayed through a display. According to one embodiment, the method may include an operation in which an expanded screen of the first execution screen is controlled through a gaze and / or gesture of a user of the wearable electronic device. According to one embodiment, the method may include an operation in which the external electronic device is controlled through a peripheral device paired with the external electronic device.

[0014] According to one embodiment, a method for controlling the wearable electronic device may be performed using a non-transitory computer-readable storage medium storing one or more programs. According to one embodiment, the one or more programs may include instructions and / or commands that are executed when executed by at least one processor of the electronic device.

[0015] According to various embodiments of the present invention, a user of a wearable electronic device can control a screen provided through a display (e.g., a display module) through gaze and / or gestures.

[0016] According to various embodiments of the present invention, when a user wears a wearable electronic device, the wearable electronic device can automatically connect, reconnect, and / or disconnect with an external electronic device and / or peripheral device within a specific range.

[0017] In addition, various effects may be provided, either directly or indirectly, through this document.

[0018] In connection with the description of the drawings, the same or similar reference numerals may be used for the same or similar components.

[0019] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments of the present invention.

[0020] FIG. 2 is a block diagram schematically showing the configuration of a wearable electronic device according to one embodiment of the present invention.

[0021] FIG. 3 is a diagram schematically illustrating an embodiment in which a wearable electronic device according to one embodiment of the present invention exchanges content with an external electronic device through mirroring.

[0022] FIG. 4 is a diagram schematically illustrating an embodiment in which a wearable electronic device according to one embodiment of the present invention exchanges content with an external electronic device through casting.

[0023] FIG. 5 is a diagram schematically illustrating an embodiment in which a wearable electronic device according to one embodiment of the present invention exchanges content with an external electronic device through streaming.

[0024] FIG. 6 is a schematic diagram illustrating an embodiment of displaying a first execution screen when a second execution screen is not displayed on a display of a wearable electronic device according to one embodiment of the present invention.

[0025] FIG. 7 is a diagram schematically illustrating an embodiment of displaying a first execution screen by moving and deforming it when a second execution screen is displayed on a display of a wearable electronic device according to one embodiment of the present invention.

[0026] FIG. 8 is a diagram schematically illustrating an embodiment in which the first execution screen and the second execution screen are displayed in a modified manner on a display of a wearable electronic device according to one embodiment of the present invention.

[0027] FIG. 9 is a drawing schematically showing a first size adjustment icon formed in a first window among the expanded screens of a first execution screen displayed on a display of a wearable electronic device according to one embodiment of the present invention.

[0028] FIG. 10 is a drawing schematically showing a second size adjustment icon formed in a second window among the expanded screens of the first execution screen displayed on the display of a wearable electronic device according to one embodiment of the present invention.

[0029] FIG. 11 is a drawing schematically showing an example in which the sizes of the first window and the second window are adjusted among the expanded screens of the first execution screen displayed on the display of a wearable electronic device according to one embodiment of the present invention.

[0030] FIG. 12 is a diagram schematically illustrating an embodiment in which the recognition range of gaze coordinates can be expanded when a first execution screen displayed on a display of a wearable electronic device according to one embodiment of the present invention includes three-dimensional content.

[0031] FIG. 13 is a diagram schematically illustrating an embodiment of displaying at least one application on a first execution screen displayed on a display of a wearable electronic device according to one embodiment of the present invention.

[0032] FIG. 14 is a diagram schematically illustrating an embodiment of popping up an application when gaze coordinates are recognized in an application displayed on a display of a wearable electronic device according to one embodiment of the present invention.

[0033] FIG. 15 is a diagram schematically illustrating an embodiment of highlighting an application displayed on a display of a wearable electronic device according to one embodiment of the present invention when gaze coordinates are recognized.

[0034] FIG. 16 is a diagram schematically illustrating an embodiment of an application displayed on a display of a wearable electronic device according to one embodiment of the present invention, in which an application is displayed in an enlarged manner when gaze coordinates are recognized.

[0035] FIG. 17 is a flowchart schematically illustrating a method for controlling a wearable electronic device according to one embodiment of the present invention.

[0036] FIG. 18 is a diagram schematically illustrating a login process of a wearable electronic device according to one embodiment of the present invention and at least one external electronic device and peripheral device.

[0037] FIG. 19 is a schematic diagram illustrating an embodiment of identifying at least one external electronic device in the vicinity of a wearable electronic device according to one embodiment of the present invention.

[0038] FIG. 20 is a schematic diagram illustrating an embodiment of pairing a wearable electronic device and at least one external electronic device according to one embodiment of the present invention.

[0039] FIG. 21 is a schematic diagram illustrating an embodiment of pairing a wearable electronic device and at least one peripheral device according to one embodiment of the present invention.

[0040] FIG. 22 is a schematic diagram illustrating an embodiment of multi-point pairing of a wearable electronic device and at least one peripheral device according to one embodiment of the present invention.

[0041] FIG. 23 is a diagram schematically illustrating an embodiment of pairing and unpairing a wearable electronic device and at least one peripheral device according to one embodiment of the present invention.

[0042] FIG. 24 is a schematic diagram illustrating an embodiment of pairing and unpairing a wearable electronic device, at least one external electronic device, and at least one peripheral device according to one embodiment of the present invention.

[0043] FIG. 25 is a schematic diagram illustrating an embodiment of pairing of a wearable electronic device when at least one external electronic device and at least one peripheral device are in use according to one embodiment of the present invention.

[0044] FIG. 26 is a schematic diagram illustrating an embodiment in which a wearable electronic device according to one embodiment of the present invention displays a virtual character input unit.

[0045] FIG. 27 is a diagram schematically illustrating an embodiment related to unpairing of a wearable electronic device, at least one external electronic device, and at least one peripheral device according to one embodiment of the present invention.

[0046] FIG. 28 is a schematic diagram illustrating an embodiment in which at least one external electronic device and at least one peripheral device are paired again according to one embodiment of the present invention.

[0047] FIG. 29 is a schematic diagram illustrating an embodiment in which at least one peripheral device according to one embodiment of the present invention fails to re-pair with at least one external electronic device.

[0048] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to various embodiments of the present invention.

[0049] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0050] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or calculations. According to one embodiment, as at least a part of the data processing or calculations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or a secondary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor)) that can operate independently or together therewith. For example, if the electronic device (101) includes a main processor (121) and a secondary processor (123), the secondary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a specified function. The secondary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0051] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0052] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0053] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0054] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0055] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0056] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. In one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0057] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0058] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0059] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0060] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0061] A haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. In one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0062] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0063] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least a part of a power management integrated circuit (PMIC).

[0064] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0065] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0066] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0067] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas by, for example, the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0068] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band.

[0069] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0070] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0071] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0072] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0073] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0074] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0075] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as a computer program product. The computer program product may be traded between sellers and buyers as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or may be provided through an application store (e.g., Play Store). TM ) or directly between two user devices (e.g., smart phones), online distribution (e.g., downloading or uploading). In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0076] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0077] FIG. 2 is a block diagram schematically showing the configuration of a wearable electronic device according to one embodiment of the present invention.

[0078] According to one embodiment, the wearable electronic device (101) disclosed below may include at least some of the embodiments described in FIG. 1. For example, the wearable electronic device (101) of FIG. 2 may include at least one component disclosed in FIG. 1. In the description of the wearable electronic device (101) disclosed below, components that are substantially the same as those of the embodiments disclosed in FIG. 1 are given the same reference numerals, and redundant descriptions of their functions may be omitted.

[0079] According to various embodiments, the wearable electronic device (101) disclosed below may include any one of augmented reality (AR) glasses (e.g., AR device), virtual reality (VR) glasses (e.g., VR device), video see through (VST) mixed reality (MR) glasses, or a head mounted display (HMD).

[0080] Referring to FIG. 2, a wearable electronic device (101) (e.g., the electronic device (101) of FIG. 1) may include a display (160), a sensor (176), a camera (180), a communication circuit (190), a memory (130), and / or a processor (120).

[0081] According to various embodiments, the display (160) may include the display module (160) disclosed in FIG. 1. The sensor (176) may include the sensor module (176) disclosed in FIG. 1. The camera (180) may include the camera module (180) disclosed in FIG. 1. For example, at least one camera (180) may be included in the wearable electronic device (101). The communication circuit (190) may include the communication module (190) disclosed in FIG. 1.

[0082] According to one embodiment, the display (160) (e.g., the display module (160) of FIG. 1) can display a screen running on the wearable electronic device (101). For example, the display (160) can display a screen related to a virtual object (e.g., virtual content) in a virtual space. The display (160) can display a screen transmitted through an external electronic device (e.g., the electronic device (102, 104) of FIG. 1 or the server (108)). For example, the display (160) can display an execution screen of a content or application running on the external electronic device (e.g., the electronic device (102, 104) of FIG. 1 or the server (108)) by expanding and / or reducing it in a virtual space.

[0083] According to various embodiments, the display (160) may display the initial screen, setting screen, and / or execution screens of various applications of the wearable electronic device (101). The display (160) may display a user interface (UI) that may provide various services to the user of the electronic device (101). The display (160) may perform display functions and input functions.

[0084] According to various embodiments, the sensor (176) (e.g., the sensor module (176) of FIG. 1) may include a wear detection sensor (211), an inertial measurement sensor (213), an infrared sensor (215), and / or a UWB sensor (217).

[0085] According to one embodiment, the wear detection sensor (211) can detect whether the user is wearing or not wearing the wearable electronic device (101) on a part of the body (e.g., face or head).

[0086] According to one embodiment, an inertial measurement unit (IMU) (213) may measure at least one of movement, posture, inclination, orientation, direction, movement speed, and gravitational acceleration of the wearable electronic device (101). The inertial measurement sensor (213) may include at least one of an acceleration sensor, a gyro sensor, an angular velocity sensor, and a geomagnetic sensor.

[0087] According to one embodiment, the infrared sensor (215) can detect movement and / or distance of an object in the vicinity of the wearable electronic device (101). For example, the infrared sensor (215) can include at least one of a proximity sensor, a light sensor, an ultrasonic sensor, and a radar sensor.

[0088] According to one embodiment, an ultra wide band (UWB) sensor (217) can detect the location of an external electronic device (e.g., an electronic device (102, 104) or a server (108) of FIG. 1) and / or a peripheral device within a specific range of a wearable electronic device (101) using UWB radio waves.

[0089] According to various embodiments, the external electronic device (e.g., the electronic device (102, 104) or the server (108) of FIG. 1) may include a smart phone, a TV, a tablet, a computer, a laptop, a set top box, and / or a game console. For example, the peripheral device may include a watch, wireless earphones, a stylus pen, a remote controller, a game pad, a keyboard, a mouse, and / or a speaker. For example, the wearable electronic device (101) may be paired with an external electronic device (e.g., the electronic device (102, 104) or the server (108) of FIG. 1) and / or a peripheral device. For example, the wearable electronic device (101) may be paired with an external electronic device (e.g., the electronic device (102, 104) or the server (108) of FIG. 1) and / or a peripheral device via a communication circuit (190). For example, a wearable electronic device (101) may be functionally connected to an external electronic device (e.g., an electronic device (102, 104) or a server (108) of FIG. 1) and / or a peripheral device via wireless communication (e.g., Bluetooth).

[0090] According to various embodiments, the camera (180) (e.g., the camera module (180) of FIG. 1) may include a depth camera (222), a gaze tracking camera (224), and / or a gesture detection camera (226). For example, at least one camera (180) may be included in the wearable electronic device (101).

[0091] According to one embodiment, the depth camera (222) can obtain image information about objects surrounding the wearable electronic device (101). The depth camera (222) can obtain two-dimensional images and / or three-dimensional images of objects outside the wearable electronic device (101).

[0092] According to one embodiment, the gaze tracking camera (224) can determine the field of view (FOV) that the user of the wearable electronic device (101) is looking at. The gaze tracking camera (224) can track the movement of the pupils (e.g., eyeballs) of the user of the wearable electronic device (101). For example, the gaze tracking camera (224) can obtain information related to the field of view and gaze coordinates toward which the eyes of the user of the wearable electronic device (101) are directed. The gaze tracking camera (224) can recognize an external electronic device (e.g., the electronic device (102, 104) or the server (108) of FIG. 1) and / or a peripheral device looking through the wearable electronic device (101).

[0093] According to various embodiments, the gaze tracking camera (224) can track the user's gaze using at least one of an EOG sensor (electro-oculography or electrooculogram), a coil system, a dual Purkinje system, bright pupil systems, or dark pupil systems.

[0094] According to one embodiment, the gesture detection camera (226) can detect movement of a part of the body of a user of the wearable electronic device (101). For example, the gesture detection camera (226) can detect information related to a gesture and / or gesture coordinates corresponding to a movement of a hand of the user of the wearable electronic device (101).

[0095] According to various embodiments, the communication circuit (190) (e.g., the communication module (190) of FIG. 1) may perform wired and / or wireless communication with an external electronic device (e.g., the electronic device (102, 104) of FIG. 1 or the server (108)) and / or a peripheral device. For example, the wearable electronic device (101) may be paired with an external electronic device (e.g., the electronic device (102, 104) of FIG. 1 or the server (108)) and / or a peripheral device using the communication circuit (190). For example, the wearable electronic device (101) may be communicatively connected with an external electronic device (e.g., the electronic device (102, 104) of FIG. 1 or the server (108)) and / or a peripheral device using the communication circuit (190). For example, a wearable electronic device (101) can exchange at least one piece of information with an external electronic device (e.g., an electronic device (102, 104) or a server (108) of FIG. 1) and / or a peripheral device using a communication circuit (190).

[0096] According to various embodiments, the external electronic device (310) (e.g., the electronic device (102, 104) or server (108) of FIG. 1) or peripheral device disclosed below may include substantially the same at least one component of the electronic device (101) disclosed in FIG. 1.

[0097] According to one embodiment, the memory (130) may store executable instructions. For example, the memory (130) may store at least one instruction that, when executed by the processor (120), causes the electronic device (101) to perform at least one operation. For example, the at least one instruction may be stored on a computer-readable recording medium. The recording medium may be tangible and non-transitory. The memory (130) and / or the recording medium may store one or more programs including at least one instruction.

[0098] According to various embodiments, the memory (130) may store the initial screen, setting screen, and / or various applications of the wearable electronic device (101) provided through the display (160). The memory (130) may store a user interface (UI) that may provide various services to the user of the wearable electronic device (101). The memory (130) may perform a function of storing a program for processing and controlling the processor (120) (e.g., the program (140) of FIG. 1), an operating system (e.g., the operating system (142) of FIG. 1), various applications, and input / output data. The memory (130) may store a program that controls the overall operation of the wearable electronic device (101). The memory (130) may store various setting information required when processing functions in the wearable electronic device (101).

[0099] According to one embodiment, the processor (120) may be operatively connected to a display (160), a sensor (176), a camera (180), a communication circuit (190), and / or a memory (130). The processor (120) may be configured to access the memory (130) and execute at least one instruction. The wearable electronic device (101) may include at least one processor (120). The wearable electronic device (101) may include at least one memory (130).

[0100] According to various embodiments, the processor (120) may control the overall operation of the wearable electronic device (101) and the signal flow between internal components, and may perform a function of processing data. The processor (120) may include, for example, a central processing unit (CPU), an application processor, and / or a communication processor. The processor (120) may include a single core processor or a multi-core processor. The processor (120) may be composed of at least one processor. For example, the processor (120) may include an application processor (e.g., the main processor (121) of FIG. 1) and a sensor hub processor (e.g., the auxiliary processor (123) of FIG. 1).

[0101] According to various embodiments, an operation executed in a wearable electronic device (101) may be performed and / or executed by at least one instruction stored in a processor (120) and / or a memory (130). For example, in this document, an embodiment in which a wearable electronic device (101) may perform a certain operation may be interpreted to mean that the operation is performed by at least one instruction stored in a processor (120) and / or a memory (130).

[0102] FIG. 3 is a diagram schematically illustrating an embodiment in which a wearable electronic device according to one embodiment of the present invention exchanges content with an external electronic device through mirroring.

[0103] Referring to FIG. 3, a wearable electronic device (101) can be paired with an external electronic device (310) (e.g., electronic devices (102, 104) or server (108) of FIG. 1) within a certain area. For example, the wearable electronic device (101) can be communicatively connected to an external electronic device (310) within a Bluetooth communication range. For example, the external electronic device (310) can be paired with a peripheral device (320) (e.g., a remote controller or a game controller). For example, the external electronic device (310) or the peripheral device (320) can include at least one embodiment and components of the electronic device (101) disclosed in FIG. 1.

[0104] According to one embodiment, the wearable electronic device (101) can recognize an external electronic device (310) that a user views through a camera (180). For example, the wearable electronic device (101) can recognize an external electronic device (310) when the user views the external electronic device (310) within a field of view (FOV) through the camera (180). For example, the wearable electronic device (101) and the external electronic device (310) may be within a communication range (e.g., a Bluetooth communication range). For example, the wearable electronic device (101) and the external electronic device (310) may be paired through a communication circuit (190).

[0105] According to one embodiment, the wearable electronic device (101) may transmit information related to the gaze coordinates through the user's eyes (301) and / or the gesture coordinates through the user's hands (302) to the external electronic device (310) when the external electronic device (310) is recognized and the user gazes at the first execution screen (315) being played on the external electronic device (310).

[0106] According to one embodiment, the external electronic device (310) may transmit the first execution screen (315) being played to the wearable electronic device (101) through mirroring (e.g., the first transmission method) based on information related to gaze coordinates and / or gesture coordinates transmitted from the wearable electronic device (101). For example, the wearable electronic device (101) may receive the first execution screen (315) being played on the external electronic device (310) through mirroring.

[0107] According to various embodiments, the first execution screen (315) may be classified as first content or second content depending on the type of content played on the external electronic device (310). For example, the type of content may be determined based on the size of the total data amount of the content. For example, the wearable electronic device (101) or the external electronic device (310) may determine the type of content based on the data transmission speed. For example, the type of content may be determined based on whether the display (160) of the wearable electronic device (101) supports a certain resolution. For example, the first execution screen (315) transmitted to the wearable electronic device (101) via mirroring (e.g., the first transmission method) from the external electronic device (310) may include the first content. For example, the first content may have a smaller total data amount, a lower data transmission speed, or a lower resolution (e.g., 4k) than the second content.

[0108] According to one embodiment, the wearable electronic device (101) can display an extended screen (350) of a first execution screen (315) received from an external electronic device (310) in a virtual space through the display (160).

[0109] According to one embodiment, the wearable electronic device (101) can control an extended screen (350) of a first execution screen (315) transmitted through an external electronic device (310) through a gaze through the user's eyes (301) and / or a gesture through the user's hands (302).

[0110] In one embodiment, an external electronic device (310) can be controlled via a paired peripheral device (320) (e.g., a remote controller or game controller).

[0111] FIG. 4 is a diagram schematically illustrating an embodiment in which a wearable electronic device according to one embodiment of the present invention exchanges content with an external electronic device through casting.

[0112] Referring to FIG. 4, a wearable electronic device (101) can be paired with an external electronic device (310) within a certain area (e.g., electronic devices (102, 104) or server (108) of FIG. 1). For example, the wearable electronic device (101) can be communicatively connected to an external electronic device (310) within a Bluetooth communication range. For example, the external electronic device (310) can be paired with a peripheral device (320) (e.g., a remote controller or a game controller). For example, the external electronic device (310) or the peripheral device (320) can include at least one embodiment and components of the electronic device (101) disclosed in FIG. 1.

[0113] According to one embodiment, the wearable electronic device (101) can recognize an external electronic device (310) that a user views through a camera (180). For example, the wearable electronic device (101) can recognize an external electronic device (310) when the user views the external electronic device (310) within a field of view (FOV) through the camera (180). For example, the wearable electronic device (101) and the external electronic device (310) may be within a communication range (e.g., a Bluetooth communication range).

[0114] According to one embodiment, the wearable electronic device (101) may transmit information related to the gaze coordinates through the user's eyes (301) and / or the gesture coordinates through the user's hands (302) to the external electronic device (310) when the external electronic device (310) is recognized and the user gazes at the first execution screen (315) being played on the external electronic device (310).

[0115] According to one embodiment, the external electronic device (310) may transmit the first execution screen (315) being played to the wearable electronic device (101) through casting (e.g., the second transmission method) based on information related to gaze coordinates and / or gesture coordinates transmitted from the wearable electronic device (101). For example, the wearable electronic device (101) may receive the first execution screen (315) being played on the external electronic device (310) through casting.

[0116] According to various embodiments, the first execution screen (315) may be classified as first content or second content depending on the type of content played on the external electronic device (310). For example, the type of content may be determined based on the size of the total data amount of the content. For example, the wearable electronic device (101) or the external electronic device (310) may determine the type of content based on the data transmission speed. For example, the type of content may be determined based on whether the display (160) of the wearable electronic device (101) supports a certain resolution. For example, the first execution screen (315) transmitted to the wearable electronic device (101) through casting (e.g., the second transmission method) from the external electronic device (310) may include second content. For example, the second content may have a larger total data amount than the first content, a high-speed data transmission speed, or a high resolution (e.g., 8k).

[0117] According to various embodiments, when the wearable electronic device (101) receives a first execution screen (315) including second content through an external electronic device (310), the wearable electronic device (101) can store a file related to the first execution screen (315) transmitted through the external electronic device (310) in real time in the memory (130).

[0118] According to various embodiments, the wearable electronic device (101) may display a file related to the first execution screen (315) including the second content stored in the memory (130) through the expanded screen (350). For example, the wearable electronic device (101) may display a warning pop-up (410) through the display (160) together with the expanded screen (350) of the first execution screen (315). For example, the warning pop-up (410) may include a warning message such as "High-definition playback is required. A slight delay may occur during playback."

[0119] According to one embodiment, the wearable electronic device (101) can control an extended screen (350) of a first execution screen (315) transmitted through an external electronic device (310) through a gaze through the user's eyes (301) and / or a gesture through the user's hands (302).

[0120] In one embodiment, an external electronic device (310) can unpair with a paired peripheral device (320) (e.g., a remote controller or game controller).

[0121] According to one embodiment, the wearable electronic device (101) is paired with a peripheral device (320) and can control an extended screen (350) of a first execution screen (315) under the control of the peripheral device (320).

[0122] FIG. 5 is a diagram schematically illustrating an embodiment in which a wearable electronic device according to one embodiment of the present invention exchanges content with an external electronic device through streaming.

[0123] Referring to FIG. 5, a wearable electronic device (101) can be paired with an external electronic device (310) (e.g., electronic devices (102, 104) or server (108) of FIG. 1) within a certain area. For example, the wearable electronic device (101) can be communicatively connected to an external electronic device (310) within a Bluetooth communication range. For example, the external electronic device (310) can be paired with a peripheral device (320) (e.g., a remote controller or a game controller). For example, the external electronic device (310) or the peripheral device (320) can include at least one embodiment and components of the electronic device (101) disclosed in FIG. 1.

[0124] According to one embodiment, the wearable electronic device (101) can recognize an external electronic device (310) that a user views through a camera (180). For example, the wearable electronic device (101) can recognize an external electronic device (310) when the user views the external electronic device (310) within a field of view (FOV) through the camera (180). For example, the wearable electronic device (101) and the external electronic device (310) may be within a communication range (e.g., a Bluetooth communication range).

[0125] According to one embodiment, the wearable electronic device (101) may transmit information related to the gaze coordinates through the user's eyes (301) and / or the gesture coordinates through the user's hands (302) to the external electronic device (310) when the external electronic device (310) is recognized and the user gazes at the first execution screen (315) running on the external electronic device (310).

[0126] According to one embodiment, the external electronic device (310) may transmit the first execution screen (315) being played to the wearable electronic device (101) through streaming (e.g., the third transmission method) based on information related to gaze coordinates and / or gesture coordinates transmitted from the wearable electronic device (101). For example, the wearable electronic device (101) may receive the first execution screen (315) being played on the external electronic device (310) through streaming.

[0127] According to various embodiments, the wearable electronic device (101) may include an application that is substantially the same as an application installed on the external electronic device (310) to execute and display the first execution screen (315). For example, the wearable electronic device (101) and the external electronic device (310) may include substantially the same application that can execute and display the first execution screen (315). For example, when the wearable electronic device (101) and the external electronic device (310) include the same application that can execute and display the first execution screen (315), the wearable electronic device (101) may receive URL information related to the first execution screen (315) from the external electronic device (310) through casting and / or streaming.

[0128] According to various embodiments, if the wearable electronic device (101) does not include an application for executing and displaying the first execution screen (315) transmitted through the external electronic device (310), the wearable electronic device (101) may display a warning pop-up (410) through the display (160). For example, the warning pop-up (410) may include a warning message such as "High-definition playback is required. Please install the corresponding app on the wearable electronic device."

[0129] According to various embodiments, a user of a wearable electronic device (101) may check the warning pop-up (410) and install an application to execute a first execution screen (315) on the wearable electronic device (101).

[0130] According to one embodiment, the wearable electronic device (101) can display information related to the first execution screen (315) transmitted from the external electronic device (101) as an extended screen (350) through the display (160).

[0131] According to one embodiment, the wearable electronic device (101) can control an extended screen (350) of a first execution screen (315) transmitted through an external electronic device (310) through a gaze through the user's eyes (301) and / or a gesture through the user's hands (302).

[0132] In one embodiment, an external electronic device (310) can unpair with a paired peripheral device (320) (e.g., a remote controller or game controller).

[0133] According to one embodiment, the wearable electronic device (101) is paired with a peripheral device (320) and can control an extended screen (350) of a first execution screen (315) under the control of the peripheral device (320).

[0134] FIG. 6 is a schematic diagram illustrating an embodiment of displaying a first execution screen when a second execution screen is not displayed on a display of a wearable electronic device according to one embodiment of the present invention.

[0135] According to various embodiments, the embodiments displaying the first execution screen (315), the expansion screen (350) and / or the second execution screen (710) disclosed below can be applied to the embodiments disclosed in FIGS. 3 to 5 described above.

[0136] Referring to FIG. 6, when a user of a wearable electronic device (101) gazes at a first execution screen (315) of an external electronic device (310) through a camera (180), the wearable electronic device (101) can display an extended screen (350) of the first execution screen (315) transmitted from the external electronic device (310) on the display (160).

[0137] According to one embodiment, when another screen (e.g., a second execution screen (e.g., the second execution screen (710) of FIG. 7)) is not displayed on the display (160) of the wearable electronic device (101), the wearable electronic device (101) may display an extended screen (350) corresponding to the first execution screen (315) transmitted from the external electronic device (310) in a virtual space on the display (160).

[0138] FIG. 7 is a diagram schematically illustrating an embodiment of displaying a first execution screen by moving and deforming it when a second execution screen is displayed on a display of a wearable electronic device according to one embodiment of the present invention.

[0139] Referring to FIG. 7, when a user gazes at a first execution screen (315) of an external electronic device (310) through a camera (180) while a second execution screen (710) is displayed on a display (160) of a wearable electronic device (101), the wearable electronic device (101) may display an extended screen (350) of the first execution screen (315) transmitted from the external electronic device (310) on the display (160). For example, the display position of the extended screen (350) may be determined depending on where the first execution screen (315) is located with respect to the center of the display (160). According to various embodiments, the display position of the extended screen (350) may be determined depending on where the second execution screen (710) is located with respect to the center of the display (160).

[0140] According to one embodiment, when a user gazes at a first execution screen (315) of an external electronic device (310) through a camera (180) while a second execution screen (710) of a wearable electronic device (101) is displayed at the center of the display (160), the wearable electronic device (101) may move the second execution screen (710) in a first direction (e.g., x-axis direction) or a second direction (e.g., -x-axis direction) and display an extended screen (350) of the first execution screen (315) transmitted from the external electronic device (310) at the center of the display (160).

[0141] According to one embodiment, when an extended screen (350) of a first execution screen (315) transmitted from an external electronic device (310) is displayed on the display (160) while a second execution screen (710) is displayed adjacent to the center of the display (160) of the wearable electronic device (101), the second execution screen (710) may be moved from a first direction (e.g., in the x-axis direction) to a second direction (e.g., in the -x-axis direction), and the extended screen (350) of the first execution screen (315) may be positioned in the first direction.

[0142] According to one embodiment, when an extended screen (350) of a first execution screen (315) transmitted from an external electronic device (310) is displayed on the display (160) while a second execution screen (710) is displayed adjacent to the center of the display (160) of the wearable electronic device (101), the second execution screen (710) may be moved from a second direction (e.g., in the -x-axis direction) to a first direction (e.g., in the x-axis direction), and the extended screen (350) of the first execution screen (315) may be positioned in the second direction.

[0143] According to one embodiment, the second execution screen (710) moved from the first direction (e.g., x-axis direction) to the second direction (e.g., -x-axis direction) may be deformed to be distinct from the extended screen (350) of the first execution screen (315). For example, the second execution screen (710) may be changed to an inclined shape (e.g., tilting shape) or displayed transparently to be distinct from the extended screen (350) of the first execution screen (315).

[0144] According to one embodiment, the second execution screen (710) moved from the second direction (e.g., -x-axis direction) to the first direction (e.g., x-axis direction) may be deformed to be distinct from the extended screen (350) of the first execution screen (315). For example, the second execution screen (710) may be changed to an inclined shape (e.g., tilting shape) or displayed transparently to be distinct from the extended screen (350) of the first execution screen (315).

[0145] FIG. 8 is a diagram schematically illustrating an embodiment in which the first execution screen and the second execution screen are displayed in a modified manner on a display of a wearable electronic device according to one embodiment of the present invention.

[0146] Referring to FIG. 8, when a user gazes at a first execution screen (315) of an external electronic device (310) through a camera (180) while a second execution screen (710) is displayed on a display (160) of a wearable electronic device (101), the wearable electronic device (101) may display an extended screen (350) of the first execution screen (315) transmitted from the external electronic device (310) on the display (160). For example, the display position of the extended screen (350) may be determined depending on where the first execution screen (315) is located with respect to the center of the display (160). According to various embodiments, the display position of the extended screen (350) may be determined depending on where the second execution screen (710) is located with respect to the center of the display (160).

[0147] According to one embodiment, when an extended screen (350) of a first execution screen (315) transmitted from an external electronic device (310) is displayed on the display (160) while a second execution screen (710) is displayed on the display (160) of a wearable electronic device (101), the wearable electronic device (101) can arrange the extended screen (350) of the first execution screen (615) and the second execution screen (710) through the display (160) so that they do not overlap.

[0148] According to one embodiment, when an extended screen (350) of a first execution screen (315) transmitted from an external electronic device (310) is displayed on the display (160) while a second execution screen (710) is displayed on the display (160) of a wearable electronic device (101), the wearable electronic device (101) may arrange the extended screen (350) of the first execution screen (615) and the second execution screen (710) in a second direction (e.g., -x-axis direction) and a first direction (e.g., x-axis direction) based on the center of the display (160).

[0149] According to one embodiment, when an extended screen (350) of a first execution screen (315) transmitted from an external electronic device (310) is displayed on the display (160) of a wearable electronic device (101) while a second execution screen (710) is displayed on the display (160), the wearable electronic device (101) may transform the extended screen (350) of the first execution screen (615) and the second execution screen (710). For example, the extended screen (350) of the first execution screen (615) and the second execution screen (710) may be transformed into an inclined shape (e.g., a tilting shape) so as to be distinguished from each other. For example, the extended screen (350) of the first execution screen (615) and the second execution screen (710) may be displayed transparently so that the extended screen (350) of the first execution screen (615) and the second execution screen (710) are distinguished from each other.

[0150] FIG. 9 is a diagram schematically illustrating a first size adjustment icon formed in a first window among the expanded screens of a first execution screen displayed on a display of a wearable electronic device according to an embodiment of the present invention. FIG. 10 is a diagram schematically illustrating a second size adjustment icon formed in a second window among the expanded screens of a first execution screen displayed on a display of a wearable electronic device according to an embodiment of the present invention. FIG. 11 is a diagram schematically illustrating an embodiment in which the sizes of the first window and the second window are adjusted among the expanded screens of the first execution screen displayed on a display of a wearable electronic device according to an embodiment of the present invention.

[0151] Referring to FIG. 9, the wearable electronic device (101) can display an extended screen of a first execution screen received from an external electronic device (310) through the display (160). For example, the extended screen of the first execution screen can include a first window (910) and / or a second window (920). For example, the first window (910) can include a larger execution screen than the second window (920). For example, the second window (920) can include a smaller screen that is larger than the first window (910). For example, the second window (920) can be displayed inside the first window (910).

[0152] According to one embodiment, the wearable electronic device (101) may display a first size adjustment icon (901) on the display (160) when the gaze coordinates of the user's eyes (301) are recognized at the corner of the first window (910). When the first size adjustment icon (901) is displayed at the corner of the first window (910), the user of the wearable electronic device (101) may adjust the size of the first window (910) by controlling the first size adjustment icon (901) through a gesture using the hand (302).

[0153] Referring to FIG. 10, the wearable electronic device (101) can display a second size adjustment icon (1001) on the display (160) when the gaze coordinates of the user's eyes (301) are recognized at the corner of the second window (920). When the second size adjustment icon (1001) is displayed at the corner of the second window (920), the user of the wearable electronic device (101) can adjust the size of the second window (920) by controlling the second size adjustment icon (1001) through a gesture using the hand (302).

[0154] Referring to FIG. 11, a user of a wearable electronic device (101) can expand the size of a first window (910) through the first size adjustment icon (901) disclosed in FIG. 9. For example, the wearable electronic device (101) can also reduce the size of a first window (910) through the first size adjustment icon (901) disclosed in FIG. 9.

[0155] According to one embodiment, the wearable electronic device (101) can expand the size of the second window (920) through the second size adjustment icon (1001) disclosed in FIG. 10. For example, the wearable electronic device (101) can also reduce the size of the second window (920) through the second size adjustment icon (1001) disclosed in FIG. 10.

[0156] According to various embodiments, the first size adjustment icon (901) and the second size adjustment icon (1001) may include different shapes. According to various embodiments, the first size adjustment icon (901) and the second size adjustment icon (1001) are not limited to the shapes disclosed in FIGS. 9 and 10 and may include various other shapes.

[0157] FIG. 12 is a diagram schematically illustrating an embodiment in which the recognition range of gaze coordinates can be expanded when a first execution screen displayed on a display of a wearable electronic device according to one embodiment of the present invention includes three-dimensional content.

[0158] Referring to FIG. 12, the wearable electronic device (101) can display an extended screen (350) of a first execution screen received from an external electronic device (310) through a display (160).

[0159] According to one embodiment, the extended screen (350) of the first execution screen received from the external electronic device (310) may include three-dimensional content (1210). For example, the three-dimensional content (1210) may include a three-dimensional image.

[0160] According to one embodiment, when the extended screen (350) of the first execution screen includes three-dimensional content (1210), the wearable electronic device (101) can expand the recognition range of the gaze coordinates through the user's eyes (301) to correspond to the three-dimensional content (1210). For example, when the three-dimensional content (1210) is displayed in a protruding manner through the display (160), the wearable electronic device (101) can allow the user to recognize or confirm the protrudingly displayed three-dimensional content (1210) through the user's eyes (301).

[0161] FIG. 13 is a diagram schematically illustrating an embodiment of displaying at least one application on a first execution screen displayed on a display of a wearable electronic device according to one embodiment of the present invention.

[0162] Referring to FIG. 13, the wearable electronic device (101) can display an extended screen (350) of a first execution screen received from an external electronic device (310) through a display (160).

[0163] According to one embodiment, the wearable electronic device (101) may display at least one application (1310) available on the wearable electronic device (101) on an extended screen (350) of a first execution screen displayed on the display (160).

[0164] According to one embodiment, at least one application (1310) may include an application exclusively usable on the wearable electronic device (101). For example, at least one application (1310) may include files and / or content created on the wearable electronic device (101).

[0165] FIG. 14 is a diagram schematically illustrating an embodiment of popping up an application when gaze coordinates are recognized in an application displayed on a display of a wearable electronic device according to one embodiment of the present invention.

[0166] Referring to FIG. 14, the wearable electronic device (101) can display, for example, a specific application (1410) among at least one application (1310) disclosed in FIG. 13 through the display (160).

[0167] According to one embodiment, a user of a wearable electronic device (101) may gaze at a specific application (1410) through his or her eyes (301) for a specified period of time.

[0168] According to one embodiment, when the wearable electronic device (101) recognizes the gaze coordinates of a specific application (1410) through the user's eyes (301), the wearable electronic device (101) may pop up the specific application (1410) whose gaze coordinates are recognized.

[0169] According to one embodiment, a user of a wearable electronic device (101) can execute a specific application (1410) that pops up by making a gesture using a hand (302).

[0170] FIG. 15 is a diagram schematically illustrating an embodiment of highlighting an application displayed on a display of a wearable electronic device according to one embodiment of the present invention when gaze coordinates are recognized.

[0171] Referring to FIG. 15, the wearable electronic device (101) can display, for example, at least one application (1310) disclosed in FIG. 13 through the display (160) and determine whether a specific application (1410) is gazed at for a specified period of time through the user's eyes (301).

[0172] According to one embodiment, when the wearable electronic device (101) recognizes the gaze coordinates of a specific application (1410) through the user's eyes (301), the wearable electronic device (101) may highlight (1510) the specific application (1410) whose gaze coordinates are recognized.

[0173] According to one embodiment, a user of a wearable electronic device (101) can execute a specific application (1410) indicated by a highlight (1510) through a gesture using a hand (302).

[0174] FIG. 16 is a diagram schematically illustrating an embodiment of an application displayed on a display of a wearable electronic device according to one embodiment of the present invention, in which an application is displayed in an enlarged manner when gaze coordinates are recognized.

[0175] Referring to FIG. 16, the wearable electronic device (101) can display, for example, at least one application (1310) disclosed in FIG. 13 through the display (160), and determine whether a specific application (e.g., a specific application (1410) of FIG. 15) is gazed at for a specified period of time through the user's eyes (301).

[0176] According to one embodiment, when the wearable electronic device (101) recognizes the gaze coordinates of a specific application (1410) through the user's eyes (301), the wearable electronic device (101) can enlarge and display the specific application for which the gaze coordinates are recognized.

[0177] According to one embodiment, a user of a wearable electronic device (101) can execute an enlarged specific application (1610) through a gesture using a hand (302).

[0178] FIG. 17 is a flowchart schematically illustrating a method for controlling a wearable electronic device according to one embodiment of the present invention.

[0179] The method disclosed below may include, for example, the embodiments disclosed in FIGS. 1 to 16. The method disclosed below may apply, for example, the embodiments disclosed in FIGS. 1 to 16. For example, the method disclosed below is not limited to the operations described below, and may perform operations related to at least some of the embodiments disclosed in FIGS. 3 to 16. In embodiments related to the method disclosed below, the operations may be performed sequentially, but may not necessarily be performed sequentially. For example, the order of the operations may be changed. For example, the order of the operations may be performed in parallel. For example, not all of the operations may be performed, but at least some of the operations may be performed. The operations disclosed below may be performed by instructions stored in the processor (120) and / or the memory (130) of the electronic device (101).

[0180] In operation 1710, the wearable electronic device (101) may be paired with an external electronic device (310) (e.g., electronic device (102, 104) or server (108) of FIG. 1) via a communication circuit (190) (e.g., communication module (190) of FIG. 1).

[0181] According to one embodiment, a wearable electronic device (101) may be communicatively connected to an external electronic device (310) within a certain area (e.g., an electronic device (102, 104) or a server (108) of FIG. 1) using a communication circuit (190) (e.g., a communication module (190) of FIG. 1).

[0182] According to one embodiment, a wearable electronic device (101) can be communicatively connected to an external electronic device (310) within a Bluetooth communication range.

[0183] According to one embodiment, an external electronic device (310) can be paired with a peripheral device (320) (e.g., a remote controller or game controller).

[0184] In operation 1720, the wearable electronic device (101) can recognize an external electronic device (310) viewed by the user through at least one camera (180).

[0185] According to one embodiment, the wearable electronic device (101) can recognize an external electronic device (310) when a user views the external electronic device (310) within a field of view (FOV) through at least one camera (180).

[0186] In operation 1730, when the user gazes at the first execution screen (315) being played on the external electronic device (310), the wearable electronic device (101) may transmit information related to the gaze coordinates through the user's eyes (301) and / or the gesture coordinates through the user's hands (302) to the external electronic device (310).

[0187] In operation 1740, the wearable electronic device (101) can receive the first execution screen (315) being played on the external electronic device (310) through mirroring.

[0188] In operation 1750, the wearable electronic device (101) can display an extended screen (350) of a first execution screen (315) received from an external electronic device (310) in a virtual space through the display (160).

[0189] In operation 1760, the wearable electronic device (101) can control an extended screen (350) of a first execution screen (315) transmitted through an external electronic device (310) through a gaze through the user's eyes (301) and / or a gesture through the user's hands (302).

[0190] In operation 1770, an external electronic device (310) can be controlled via a paired peripheral device (320) (e.g., a remote controller or game controller).

[0191] FIG. 18 is a diagram schematically illustrating a login process of a wearable electronic device according to one embodiment of the present invention and at least one external electronic device and peripheral device.

[0192] According to various embodiments, the embodiments disclosed below may include at least one of the embodiments disclosed in FIGS. 1 to 17.

[0193] Referring to FIG. 18, when a user wears a wearable electronic device (101), the wearable electronic device (101) can recognize the iris of the user's eye (301) through at least one camera (180) and log in to a user account. For example, the wearable electronic device (101) can be pre-registered by the user through the user's iris.

[0194] According to one embodiment, a first external electronic device (1810) (e.g., a mobile phone), a second external electronic device (1820) (e.g., a laptop), a third external electronic device (1830) (e.g., a tablet), and / or a fourth external electronic device (1840) (e.g., a TV, a set-top box, or a game console) may be present around the wearable electronic device (101). For example, the wearable electronic device (101), the first external electronic device (1810), the second external electronic device (1820), the third external electronic device (1830), and / or the fourth external electronic device (1840) may be present within a range where they can be communicatively paired via Bluetooth communication. For example, the wearable electronic device (101), the first external electronic device (1810), the second external electronic device (1820), the third external electronic device (1830), and / or the fourth external electronic device (1840) may include a device that has a history of being logged in with a user account.

[0195] In one embodiment, a first external electronic device (1810) (e.g., a mobile phone) can be paired with a first-first peripheral device (1811) (e.g., a keyboard), a first-second peripheral device (1813) (e.g., wireless earphones), and / or a first-third peripheral device (1815) (e.g., a watch). For example, a second external electronic device (1820) (e.g., a laptop) can be paired with a second-first peripheral device (1821) (e.g., a keyboard) and / or a second-second peripheral device (1823) (e.g., a mouse). For example, a third external electronic device (1830) (e.g., a tablet) can be paired with a third-first peripheral device (1831) (e.g., a stylus pen). For example, a fourth external electronic device (1840) (e.g., a TV, a set-top box, or a game console) may be paired with a fourth-first peripheral device (1841) (e.g., a remote controller) and / or a fourth-second peripheral device (1843) (e.g., a game pad).

[0196] According to various embodiments, the first external electronic device (1810), the second external electronic device (1820), the third external electronic device (1830), and / or the fourth external electronic device (1840) may include at least one embodiment and component of the electronic device (101) disclosed in FIG. 1.

[0197] According to various embodiments, the first-first peripheral device (1811) (e.g., keyboard), the first-second peripheral device (1813) (e.g., wireless earphones), the first-third peripheral device (1815) (e.g., watch), the second-first peripheral device (1821) (e.g., keyboard), the second-second peripheral device (1823) (e.g., mouse), the third-first peripheral device (1831) (e.g., stylus pen), the fourth-first peripheral device (1841) (e.g., remote controller) and / or the fourth-second peripheral device (1843) (e.g., game pad) may include at least one embodiment and component of the electronic device (101) disclosed in FIG. 1.

[0198] FIG. 19 is a schematic diagram illustrating an embodiment of identifying at least one external electronic device in the vicinity of a wearable electronic device according to one embodiment of the present invention.

[0199] Referring to FIG. 19, the wearable electronic device (101) can determine whether a first external electronic device (1810), a second external electronic device (1820), a third external electronic device (1830), and / or a fourth external electronic device (1840) that has a history of being logged in through a user account is within a certain area. For example, the wearable electronic device (101) can determine whether a first external electronic device (1810), a second external electronic device (1820), a third external electronic device (1830), and / or a fourth external electronic device (1840) that has a history of being logged in through a user account is within a Bluetooth communication range.

[0200] FIG. 20 is a schematic diagram illustrating an embodiment of pairing a wearable electronic device and at least one external electronic device according to one embodiment of the present invention.

[0201] Referring to FIG. 20, when a first external electronic device (1810), a second external electronic device (1820), a third external electronic device (1830), and / or a fourth external electronic device (1840) that has a history of being logged in through a user account is identified within a certain area of ​​the wearable electronic device (101), the wearable electronic device (101) can be paired with the first external electronic device (1810), the second external electronic device (1820), the third external electronic device (1830), and / or the fourth external electronic device (1840).

[0202] FIG. 21 is a schematic diagram illustrating an embodiment of pairing a wearable electronic device and at least one peripheral device according to one embodiment of the present invention.

[0203] According to one embodiment, the first external electronic device (1810) can be paired with the 1-1 peripheral device (1811) (e.g., a keyboard), the 1-2 peripheral device (1813) (e.g., wireless earphones), and / or the 1-3 peripheral device (1815) (e.g., a watch). For example, the second external electronic device (1820) can be paired with the 2-1 peripheral device (1821) (e.g., a keyboard) and / or the 2-2 peripheral device (1823) (e.g., a mouse). For example, the 3rd external electronic device (1830) can be paired with the 3-1 peripheral device (1831) (e.g., a stylus pen). For example, a fourth external electronic device (1840) may be paired with a fourth-first peripheral device (1841) (e.g., a remote controller) and / or a fourth-second peripheral device (1843) (e.g., a game pad).

[0204] According to one embodiment, the wearable electronic device (101) can be multi-paired with a first peripheral device (1811) (e.g., a keyboard), a first peripheral device (1813) (e.g., wireless earphones) and / or a first peripheral device (1815) (e.g., a watch) paired with a first external electronic device (1810).

[0205] According to one embodiment, the wearable electronic device (101) can be multi-paired with a second-first peripheral device (1821) (e.g., a keyboard) and / or a second-second peripheral device (1823) (e.g., a mouse) paired with a second external electronic device (1820).

[0206] According to one embodiment, a wearable electronic device (101) can be multi-paired with a third-first peripheral device (1831) (e.g., a stylus pen) paired with a third external electronic device (1830).

[0207] According to one embodiment, the wearable electronic device (101) can be multi-paired with a 4-1 peripheral device (1841) (e.g., a remote controller) and / or a 4-2 peripheral device (1843) (e.g., a game pad) paired with a 4th external electronic device (1840).

[0208] FIG. 22 is a schematic diagram illustrating an embodiment of multi-point pairing of a wearable electronic device and at least one peripheral device according to one embodiment of the present invention.

[0209] According to one embodiment, the wearable electronic device (101) can check the profile connection type of the first-first peripheral device (1811) (e.g., keyboard), the first-second peripheral device (1813) (e.g., wireless earphones), the first-third peripheral device (1815) (e.g., watch), the second-first peripheral device (1821) (e.g., keyboard), the second-second peripheral device (1823) (e.g., mouse), the third-first peripheral device (1831) (e.g., stylus pen), the fourth-first peripheral device (1841) (e.g., remote controller) and / or the fourth-second peripheral device (1843) (e.g., game pad).

[0210] According to one embodiment, it is possible to determine whether there is a peripheral device used immediately before the user wears the wearable electronic device (101). For example, if there is a 1-1 peripheral device (1811) (e.g., a keyboard) and a 1-3 peripheral device (1815) (e.g., a watch) paired with a first external electronic device (1810) immediately before the user wears the wearable electronic device (101), the 1-1 peripheral device (1811) and the 1-3 peripheral device (1815) can be unpaired from the first external electronic device (1810) and paired with the wearable electronic device (101) in a multi-point manner.

[0211] According to various embodiments, the profile connection type of the at least one peripheral device may include Hands Free Profile (HFP), Stereo Audio Profile (A2DP), Human Interface Device Profile (HID), AV Remote Control Profile (AVRCP), Video Distribution Profile (VDP), Health Device Profile (HDP), Basic Printing Profile (BPP), and / or File Transfer Profile (FTP).

[0212] FIG. 23 is a diagram schematically illustrating an embodiment of pairing and unpairing a wearable electronic device and at least one peripheral device according to one embodiment of the present invention.

[0213] In one embodiment, while the wearable electronic device (101) is paired with a first external electronic device (1810) and a first-first peripheral device (1811) (e.g., a keyboard), a user of the wearable electronic device (101) may gaze at a second external electronic device (1820) (e.g., a laptop). For example, the wearable electronic device (101) may recognize the second external electronic device (1820) gazed at by the user through his or her eyes (301).

[0214] According to one embodiment, the wearable electronic device (101) can check the profile connection type of the 1-1 peripheral device (1811) (e.g., a keyboard) and the profile connection type of the 2-1 peripheral device (1821) (e.g., a keyboard) paired with the 2nd external electronic device (1820) (e.g., a laptop). For example, the 1-1 peripheral device (1811) (e.g., a keyboard) can use a human interface device profile (HID). For example, the 2-1 peripheral device (1821) (e.g., a keyboard) can use the same human interface device profile (HID) as the 1-1 peripheral device (1811) (e.g., a keyboard).

[0215] According to one embodiment, when the profile (e.g., human interface device profile (HID)) of the 1-1 peripheral device (1811) (e.g., keyboard) and the profile (e.g., human interface device profile (HID)) of the 2-1 peripheral device (1821) (e.g., keyboard) are the same, the wearable electronic device (101) may be unpaired with the 1-1 peripheral device (1811) (e.g., keyboard) and may be maintained paired with the 2-1 peripheral device (1821) (e.g., keyboard).

[0216] FIG. 24 is a diagram schematically illustrating an embodiment of pairing and unpairing a wearable electronic device, at least one external electronic device, and at least one peripheral device according to one embodiment of the present invention.

[0217] According to one embodiment, when the wearable electronic device (101) is paired with the first external electronic device (1810), the user of the wearable electronic device (101) may gaze at the second external electronic device (1820), the third external electronic device (1830), or the fourth external electronic device (1840). For example, the wearable electronic device (101) may recognize the second external electronic device (1820), the third external electronic device (1830), or the fourth external electronic device (1840) that the user gazes at through his / her eyes (301).

[0218] According to one embodiment, when a user of a wearable electronic device (101) gazes at a second external electronic device (1820), the second-first peripheral device (1821) and the second-second peripheral device (1823) may remain connected to the second external electronic device (1820) or may be paired with the wearable electronic device (101).

[0219] According to one embodiment, when a user of a wearable electronic device (101) gazes at a third external electronic device (1830), the third-first peripheral device (1831) may remain connected to the third external electronic device (1830) or may be paired with the wearable electronic device (101).

[0220] According to one embodiment, when a user of a wearable electronic device (101) gazes at a fourth external electronic device (1840), the fourth-first peripheral device (1841) and the fourth-second peripheral device (1843) may remain connected to the fourth external electronic device (1840) or may be paired with the wearable electronic device (101).

[0221] FIG. 25 is a schematic diagram illustrating an embodiment of pairing of a wearable electronic device when at least one external electronic device and at least one peripheral device are in use according to one embodiment of the present invention.

[0222] In one embodiment, a user of a wearable electronic device (101) may gaze at a second external electronic device (1820). For example, the wearable electronic device (101) may recognize the second external electronic device (1820) gazed at by the user through his or her eyes (301).

[0223] According to one embodiment, when a user of a wearable electronic device (101) gazes at a second external electronic device (1820), the second external electronic device (1820) may be in use using a second-first peripheral device (1821) and a second-second peripheral device (1823).

[0224] According to one embodiment, when the second external electronic device (1820) is used through the 2-1 peripheral device (1821) and the 2-2 peripheral device (1823), the 2-1 peripheral device (1821) and the 2-2 peripheral device (1823) may not be paired with the wearable electronic device (101), and may remain paired with the second external electronic device (1820).

[0225] FIG. 26 is a schematic diagram illustrating an embodiment in which a wearable electronic device according to one embodiment of the present invention displays a virtual character input unit.

[0226] In one embodiment, a user of a wearable electronic device (101) may gaze at a first external electronic device (1810). For example, the wearable electronic device (101) may recognize the first external electronic device (1810) gazed at by the user through his or her eyes (301).

[0227] According to one embodiment, a wearable electronic device (101) can be paired with a first external electronic device (1810).

[0228] According to one embodiment, when there is no first-first peripheral device (1811) (e.g., a keyboard) paired to the first external electronic device (1810), the wearable electronic device (101) may display a virtual character input device (105) (e.g., a virtual keyboard) through the display (160).

[0229] According to one embodiment, when there is no first-first peripheral device (1811) (e.g., a keyboard) paired to the first external electronic device (1810), the wearable electronic device (101) can activate a virtual character input device (105) (e.g., a virtual keyboard) in the virtual space through the display (160).

[0230] FIG. 27 is a diagram schematically illustrating an embodiment related to unpairing of a wearable electronic device, at least one external electronic device, and at least one peripheral device according to one embodiment of the present invention.

[0231] In one embodiment, the user may detach the wearable electronic device (101). For example, the user may not use the wearable electronic device (101).

[0232] According to one embodiment, when a user does not use the wearable electronic device (101), the wearable electronic device (101) and the first external electronic device (1810), the second external electronic device (1820), the third external electronic device (1830), and the fourth external electronic device (1840) may be unpaired.

[0233] According to one embodiment, when a user does not use the wearable electronic device (101), the wearable electronic device (101) and the first-first peripheral device (1811), the first-second peripheral device (1813), the first-third peripheral device (1815), the second-first peripheral device (1821), the second-second peripheral device (1823), the third-first peripheral device (1831), the fourth-first peripheral device (1841), and the fourth-second peripheral device (1843) may be unpaired.

[0234] FIG. 28 is a schematic diagram illustrating an embodiment in which at least one external electronic device and at least one peripheral device are paired again according to one embodiment of the present invention.

[0235] In one embodiment, the user may detach the wearable electronic device (101). For example, the user may not use the wearable electronic device (101).

[0236] In one embodiment, when the user is not using the wearable electronic device (101), at least one of the first peripheral device (1811), the first peripheral device (1813), and the first peripheral device (1815) may be re-paired with the first external electronic device (1810) that was paired before the wearable electronic device (101) was worn.

[0237] In one embodiment, when the user is not using the wearable electronic device (101), at least one of the second-first peripheral device (1821) and the second-second peripheral device (1823) can be re-paired with the second external electronic device (1820) that was paired before the wearable electronic device (101) was worn.

[0238] In one embodiment, when the user is not using the wearable electronic device (101), the 3-1 peripheral device (1831) can be re-paired with the third external electronic device (1830) that was paired before the wearable electronic device (101) was worn.

[0239] In one embodiment, when the user is not using the wearable electronic device (101), at least one of the 4-1 peripheral device (1841) and the 4-2 peripheral device (1843) can be re-paired with the fourth external electronic device (1840) that was paired before the wearable electronic device (101) was worn.

[0240] FIG. 29 is a diagram schematically illustrating an embodiment in which at least one peripheral device according to one embodiment of the present invention fails to re-pair with at least one external electronic device.

[0241] In one embodiment, if the 4-2 peripheral device (1843) (e.g., game pad) is not within a certain range of the 4th external electronic device (1840), the 4-2 peripheral device (1843) (e.g., game pad) may not be re-paired with the 4th external electronic device (1840).

[0242] According to one embodiment, if the 4-2 peripheral device (1843) (e.g., game pad) is not within a certain area with respect to the 4th external electronic device (1840), the 4th external electronic device (1840) may provide a notification screen (2030) indicating the location of the 4-2 peripheral device (1843) (e.g., game pad) immediately after the user detaches the wearable electronic device (101).

[0243] A wearable electronic device (101) according to one embodiment of the present invention may include a display (160), at least one camera (180), a communication circuit (190), at least one processor (120), and a memory (130) that stores instructions. According to one embodiment, the instructions, when executed by the at least one processor (120), may cause the wearable electronic device (101) to perform pairing with an external electronic device (310) through the communication circuit (120), recognize the external electronic device (320) viewed through the at least one camera (180), transmit information related to gaze coordinates and / or gesture coordinates of a user of the wearable electronic device (101) to the external electronic device (310), and, based on the information related to the gaze coordinates and / or gesture coordinates, receive a first execution screen (315) being played on the external electronic device (310) from the external electronic device (310) through mirroring, and display an expanded screen (350) of the first execution screen (315) received from the external electronic device (310) through the display (160). According to one embodiment, the extended screen (350) of the first execution screen (315) may be configured to be controllable through the user's gaze and / or gesture of the wearable electronic device (101). According to one embodiment, the external electronic device (310) may be configured to be controllable through a paired peripheral device (320).

[0244] According to one embodiment, the instructions, when executed by the at least one processor (120), may cause the wearable electronic device (101) to display an extended screen (350) of the first execution screen (315) on the display (160) when the second execution screen (710) is not displayed on the display (160).

[0245] According to one embodiment, the instructions, when executed by the at least one processor (120), may cause the wearable electronic device (101) to, when the second execution screen (710) is displayed on the display (160), cause the extended screen (350) of the first execution screen (315) to be displayed on the display (160), move the second execution screen (710) from the first direction to the second direction, and cause the extended screen (350) of the first execution screen (315) to be positioned in the first direction.

[0246] According to one embodiment, the instructions, when executed by the at least one processor (120), may cause the wearable electronic device (101) to transform the second execution screen (720) moved from the first direction to the second direction so as to be distinct from the extended screen (350) of the first execution screen (315).

[0247] According to one embodiment, the instructions, when executed by the at least one processor (120), may cause the wearable electronic device (101) to, when the second execution screen (710) is displayed on the display (160), cause the expansion screen (350) of the first execution screen (315) and the second execution screen (710) to be arranged so as not to overlap each other, and cause the expansion screen (350) of the first execution screen (315) and the second execution screen (710) to be deformed.

[0248] According to one embodiment, the instructions, when executed by the at least one processor (120), may cause the wearable electronic device (101) to display a first size adjustment icon (901) for adjusting the size of the first window (910) through the gesture when the gaze coordinates are recognized at a corner of the first window (910) among the extended screen (350) of the first execution screen (315) displayed on the display (160), and to display a second size adjustment icon (1001) for adjusting the size of the second window (920) through the gesture when the gaze coordinates are recognized at a corner of the second window (920) among the extended screen (350) of the first execution screen (315).

[0249] According to one embodiment, the instructions, when executed by the at least one processor (120), may cause the wearable electronic device (101) to expand the recognition range of the gaze coordinates to correspond to the three-dimensional content (1210), when the extended screen (350) of the first execution screen (315) displayed on the display (160) includes the three-dimensional content (1210).

[0250] According to one embodiment, the instructions, when executed by the at least one processor (120), may cause the wearable electronic device (101) to display at least one application (1310) available in the wearable electronic device (101) on an extended screen (350) of the first execution screen (315) displayed on the display (160).

[0251] According to one embodiment, the instructions, when executed by the at least one processor (120), may cause the wearable electronic device (101) to pop up, highlight, or enlarge the at least one application (1310) when the gaze coordinates are recognized for a specified period of time.

[0252] According to one embodiment, the instructions, when executed by the at least one processor (120), may cause the wearable electronic device (101) to check the type of content of the first execution screen (315) being played on the external electronic device (310), receive the first content from the external electronic device (310) through mirroring if the first execution screen (315) includes the first content, receive the second content from the external electronic device (310) through casting if the first execution screen (315) includes the second content, release the pairing of the external electronic device (310) and the peripheral device (320), and pair the wearable electronic device (101) and the peripheral device (320).

[0253] A method for controlling a wearable electronic device (101) according to one embodiment of the present invention may include an operation in which the wearable electronic device (101) performs pairing with an external electronic device (310) through a communication circuit (190). According to one embodiment, the method may include an operation in which the wearable electronic device (101) recognizes the external electronic device (320). According to one embodiment, the method may include an operation in which information related to gaze coordinates and / or gesture coordinates of a user of the wearable electronic device (101) is transmitted to the external electronic device (310). According to one embodiment, the method may include an operation in which a first execution screen (315) being played on the external electronic device (310) is received from the external electronic device (310) through mirroring based on the information related to the gaze coordinates and / or gesture coordinates. According to one embodiment, the method may include an operation of displaying an extended screen (350) of the first execution screen (315) received from the external electronic device (310) through the display (160). According to one embodiment, the method may include an operation of controlling the extended screen (350) of the first execution screen (315) through a gaze and / or gesture of a user of the wearable electronic device (101). According to one embodiment, the method may include an operation of controlling the external electronic device (310) through a peripheral device (320) paired with the external electronic device (310).

[0254] According to one embodiment, the method may further include an operation of displaying an extended screen (350) of the first execution screen (315) on the display (160) when the second execution screen (710) is not displayed on the display (160).

[0255] According to one embodiment, the method may further include, when the second execution screen (710) is displayed on the display (160), and the expanded screen (350) of the first execution screen (315) is displayed on the display (160), moving the second execution screen (710) from the first direction to the second direction and positioning the expanded screen (350) of the first execution screen (315) in the first direction.

[0256] According to one embodiment, the method may further include an operation of causing the second execution screen (720) moved from the first direction to the second direction to be deformed to be distinct from the extended screen (350) of the first execution screen (315).

[0257] According to one embodiment, the method may further include, when the second execution screen (710) is displayed on the display (160), an operation of arranging the expanded screen (350) of the first execution screen (315) and the second execution screen (710) so as not to overlap each other, and an operation of causing the expanded screen (350) of the first execution screen (315) and the second execution screen (710) to be deformed.

[0258] According to one embodiment, the method may further include an operation of displaying a first size adjustment icon (901) for adjusting the size of the first window (910) through the gesture when the gaze coordinates are recognized at a corner of the first window (910) among the extended screen (350) of the first execution screen (315) displayed on the display (160), and an operation of displaying a second size adjustment icon (1001) for adjusting the size of the second window (920) through the gesture when the gaze coordinates are recognized at a corner of the second window (920) among the extended screen (350) of the first execution screen (315).

[0259] According to one embodiment, the method may further include an operation of expanding the recognition range of the gaze coordinates to correspond to the three-dimensional content (1210), when the extended screen (350) of the first execution screen (315) displayed on the display (160) includes the three-dimensional content (1210).

[0260] According to one embodiment, the method may further include an operation of displaying at least one application (1310) available on the wearable electronic device (101) on an extended screen (350) of the first execution screen (315) displayed on the display (160).

[0261] According to one embodiment, the method may further include an action of popping up, highlighting, or enlarging the at least one application when the gaze coordinates are recognized for a specified period of time in the at least one application (1310).

[0262] According to one embodiment, the method may further include an operation of checking the type of content of the first execution screen (315) being played on the external electronic device (310), an operation of receiving the first content from the external electronic device (310) through mirroring when the first execution screen (315) includes the first content, an operation of receiving the second content from the external electronic device (310) through casting when the first execution screen (315) includes the second content, an operation of releasing the pairing of the external electronic device (310) and the peripheral device (320), and an operation of pairing the wearable electronic device (101) and the peripheral device (320).

[0263] Although the present invention has been described above according to various embodiments of the present invention, it is obvious that changes and modifications made by a person having ordinary skill in the art to which the present invention pertains within a scope that does not deviate from the technical characteristics of the present invention also belong to the present invention.

Claims

1. In a wearable electronic device (101), display (160); At least one camera (180); Communication circuit (190); at least one processor (120); and Includes a memory (130) for storing instructions, The above instructions, when executed by the at least one processor (120), cause the wearable electronic device (101) to: Pairing is performed with an external electronic device (310) through the above communication circuit (190), Recognize the external electronic device (320) viewed through at least one camera (180), Transmitting information related to the gaze coordinates and / or gesture coordinates of the user of the wearable electronic device (101) to the external electronic device (310), Based on the information related to the above gaze coordinates and / or gesture coordinates, the first execution screen (315) being played on the external electronic device (310) is received from the external electronic device (310) through mirroring, To display an extended screen (350) of the first execution screen (315) received from the external electronic device (310) through the display (160), The extended screen (350) of the first execution screen (315) is configured to be controllable through the user's gaze and / or gesture of the wearable electronic device (101). The above external electronic device (310) is a wearable electronic device configured to be controllable through a paired peripheral device (320).

2. In paragraph 1, The above instructions, when executed by the at least one processor (120), cause the wearable electronic device (101) to: A wearable electronic device that causes an extended screen (350) of the first execution screen (315) to be displayed on the display (160) when the second execution screen (710) is not displayed on the display (160).

3. In paragraph 1, The above instructions, when executed by the at least one processor (120), cause the wearable electronic device (101) to: While the second execution screen (710) is displayed on the above display (160), if the extended screen (350) of the first execution screen (315) is displayed on the above display (160), The above second execution screen (710) is moved from the first direction to the second direction, and the extended screen (350) of the above first execution screen (315) is positioned in the first direction. A wearable electronic device in which a second execution screen (720) moved from the first direction to the second direction is deformed to be distinct from an extended screen (350) of the first execution screen (315).

4. In paragraph 1, The above instructions, when executed by the at least one processor (120), cause the wearable electronic device (101) to: While the second execution screen (710) is displayed on the above display (160), if the extended screen (350) of the first execution screen (315) is displayed on the above display (160), The extended screen (350) of the first execution screen (315) and the second execution screen (710) are arranged so as not to overlap each other, A wearable electronic device that allows the extended screen (350) of the first execution screen (315) and the second execution screen (710) to be transformed.

5. In paragraph 1, The above instructions, when executed by the at least one processor (120), cause the wearable electronic device (101) to: When the gaze coordinates are recognized at the corner of the first window (910) among the extended screens (350) of the first execution screen (315) displayed on the display (160), the first size adjustment icon (901) for adjusting the size of the first window (910) through the gesture is displayed, A wearable electronic device that displays a second size adjustment icon (1001) for adjusting the size of the second window (920) through the gesture when the gaze coordinates are recognized at the corner of the second window (920) among the extended screens (350) of the first execution screen (315).

6. In paragraph 1, The above instructions, when executed by the at least one processor (120), cause the wearable electronic device (101) to: A wearable electronic device that expands the recognition range of the gaze coordinates to correspond to the three-dimensional content (1210) when the extended screen (350) of the first execution screen (315) displayed on the display (160) includes three-dimensional content (1210).

7. In paragraph 1, The above instructions, when executed by the at least one processor (120), cause the wearable electronic device (101) to: On the extended screen (350) of the first execution screen (315) displayed on the display (160), at least one application (1310) available on the wearable electronic device (101) is displayed, A wearable electronic device that causes at least one application (1310) to pop up, highlight, or enlarge when the gaze coordinates are recognized for a specified period of time.

8. In paragraph 1, The above instructions, when executed by the at least one processor (120), cause the wearable electronic device (101) to: Check the type of content of the first execution screen (315) being played on the external electronic device (310), If the first execution screen (315) includes the first content, the first content is received from the external electronic device (310) through the mirroring, If the first execution screen (315) above includes second content, the second content is received from the external electronic device (310) through casting, Unpair the external electronic device (310) and the peripheral device (320), A wearable electronic device that allows pairing of the above wearable electronic device (101) and the above peripheral device (320).

9. A method for controlling a wearable electronic device (101), An operation in which the above wearable electronic device (101) performs pairing with an external electronic device (310) through a communication circuit (190); An operation in which the above wearable electronic device (101) recognizes an external electronic device (320); An action of transmitting information related to the gaze coordinates and / or gesture coordinates of the user of the wearable electronic device (101) to the external electronic device (310); An operation of receiving a first execution screen (315) being played on the external electronic device (310) from the external electronic device (310) through mirroring based on information related to the above gaze coordinates and / or gesture coordinates; An operation of displaying an extended screen (350) of the first execution screen (315) received from the external electronic device (310) through a display (160); An operation of controlling an extended screen (350) of the first execution screen (315) through a user's gaze and / or gesture of the wearable electronic device (101); and A method including an operation of controlling an external electronic device (310) through a peripheral device (320) paired with the external electronic device (310).

10. In paragraph 9, A method further comprising an action of displaying an extended screen (350) of the first execution screen (315) on the display (160) when the second execution screen (710) is not displayed on the display (160).

11. In paragraph 9, While the second execution screen (710) is displayed on the above display (160), if the extended screen (350) of the first execution screen (315) is displayed on the above display (160), An operation of moving the second execution screen (710) from the first direction to the second direction and positioning the extended screen (350) of the first execution screen (315) in the first direction; and A method further comprising an action of causing a second execution screen (720) moved from the first direction to the second direction to be transformed so as to be distinct from an extended screen (350) of the first execution screen (315).

12. In paragraph 9, While the second execution screen (710) is displayed on the above display (160), if the extended screen (350) of the first execution screen (315) is displayed on the above display (160), An operation to arrange the extended screen (350) of the first execution screen (315) and the second execution screen (710) so that they do not overlap; and A method further comprising an action of causing the extended screen (350) of the first execution screen (315) and the second execution screen (710) to be transformed.

13. In paragraph 9, An operation of displaying a first size adjustment icon (901) for adjusting the size of the first window (910) through the gesture when the gaze coordinate is recognized at the corner of the first window (910) among the extended screens (350) of the first execution screen (315) displayed on the display (160); and A method further comprising an action of displaying a second size adjustment icon (1001) for adjusting the size of the second window (920) through the gesture when the gaze coordinates are recognized at a corner of the second window (920) among the extended screens (350) of the first execution screen (315).

14. In paragraph 9, A method further comprising an operation of expanding the recognition range of the gaze coordinates to correspond to the three-dimensional content (1210), when the extended screen (350) of the first execution screen (315) displayed on the display (160) includes three-dimensional content (1210).

15. In paragraph 9, An operation of displaying at least one application (1310) available on the wearable electronic device (101) on an extended screen (350) of the first execution screen (315) displayed on the display (160); and A method further comprising an action of popping up, highlighting, or enlarging at least one application when the gaze coordinates are recognized for a specified period of time in at least one application (1310).

Images