Electronic device for controlling display of virtual device, method thereof, and non-transitory computer readable storage medium thereof

Abstract

The electronic device may include a microphone, communication circuitry, and a processor. The processor may be configured to recognize, based on recognition of the voice signal acquired through the microphone, a voice command for display through a display of a virtual device registered by using a software application for controlling another device through the electronic device. The processor may be configured to identify, based on the voice command, whether a mixed reality (MR) device configured to display a virtual device is worn. The processor may be configured to acquire an image to be displayed on a display of a virtual device displayed through the MR device by using device information registered by the soft application while interacting with the virtual device in response to recognition of a wear of the MR device. The processor may be configured to transmit, via the communication circuitry, a signal to the MR device for displaying an image on a display of a virtual device displayed by the MR device.

Description

Technical Field

[0001] The present disclosure relates to electronic devices, methods, and non-transitory computer-readable storage media for controlling display of virtual devices. Background Art

[0002] Electronic devices that provide augmented reality (AR) services, mixed reality (MR) services, or extended reality (XR) services are being developed to provide an enhanced user experience, and the augmented reality (AR) services, mixed reality (MR) services, or extended reality (XR) services are used to display computer-generated information associated with real objects in the real world or visual objects corresponding to real objects. The electronic device may be a wearable device that can be worn by a user. For example, the electronic device may be AR glasses or a visual see-through (VST) device. Summary of the invention

[0003] Technical Solution

[0004] An electronic device is provided. The electronic device may include a microphone. The electronic device may include a communication circuit. The electronic device may include a processor. The processor may be configured to: based on the recognition of a voice signal obtained through the microphone, recognize a voice command, the voice command indicating the display through a display of a virtual device, the virtual device being registered by using a software application for controlling another device through the electronic device. The processor may be configured to: based on the voice command, recognize whether a mixed reality MR device for displaying the virtual device is worn. The processor may be configured to: in response to recognizing that the MR device is worn, based on device information registered as associated with the virtual device by using the software application, obtain an image to be displayed on the display of the virtual device displayed by the MR device. The processor may be configured to: send a signal to the MR device through the communication circuit for displaying the image on the display of the virtual device displayed by the MR device.

[0005] A method is provided. The method may be performed in an electronic device including a microphone and a communication circuit. The method may include: based on recognition of a voice signal obtained through the microphone, recognizing a voice command, the voice command indicating display through a display of a virtual device, the virtual device being registered by using a software application for controlling another device through the electronic device. The method may include: based on the voice command, recognizing whether a mixed reality MR device for displaying the virtual device is worn. The method may include: in response to recognizing that the MR device is worn, based on device information registered as associated with the virtual device by using the software application, obtaining an image to be displayed on the display of the virtual device displayed by the MR device. The method may include: sending a signal to the MR device through the communication circuit for displaying the image on the display of the virtual device displayed by the MR device.

[0006] A non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium may store one or more programs. The one or more programs may include instructions, which, when executed by a processor of an electronic device including a microphone and a communication circuit, causes the electronic device to: recognize a voice command based on recognition of a voice signal obtained through the microphone, the voice command indicating display through a display of a virtual device, the virtual device being registered by using a software application for controlling another device through the electronic device. The one or more programs may include instructions, which, when executed by the processor, causes the electronic device to: recognize whether a mixed reality MR device for displaying the virtual device is worn based on the voice command. The one or more programs may include instructions, which, when executed by the processor, causes the electronic device to: obtain an image to be displayed on the display of the virtual device displayed by the MR device based on device information registered as associated with the virtual device by using the software application in response to recognizing that the MR device is worn. The one or more programs may include instructions, which, when executed by the processor, causes the electronic device to: send a signal to the MR device through the communication circuit for displaying the image on the display of the virtual device displayed by the MR device. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The above and other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent through the following detailed description in conjunction with the accompanying drawings, in which:

[0008] Figure 1 An example of an environment including an exemplary electronic device and a mixed reality (MR) device is shown.

[0009] Figure 2 is a simplified block diagram of an exemplary electronic device.

[0010] Figure 3 is a flow chart illustrating an exemplary method of registering a virtual device.

[0011] Figure 4a An exemplary method of registering a virtual device by using a software application for controlling another device is shown.

[0012] Figure 4b An example of a user interface including a visual object corresponding to a virtual device registered by using a software application for controlling another device is shown.

[0013] Figure 5 is a flow chart illustrating an exemplary method of displaying an image through a display of a virtual device.

[0014] Figure 6 is a flow chart illustrating an exemplary method of recognizing a voice command based on a display location.

[0015] Figure 7 is a flow chart illustrating an exemplary method of obtaining an image to be displayed by a display of a virtual device based on information obtained from another electronic device.

[0016] Figure 8 is a flow chart illustrating an exemplary method of recognizing a voice command based on a location of an electronic device, a location of another electronic device, and a location of a virtual device.

[0017] Fig. 9 is a flowchart illustrating an exemplary method of obtaining an image based on device information of a virtual device transmitted to an external electronic device.

[0018] Fig.10 is a flow chart illustrating an exemplary method of recognizing a voice command based on an orientation of a display of another electronic device and an orientation of a display of a virtual device.

[0019] Fig.11 is a flow chart illustrating an exemplary method of controlling a virtual device located in a virtual space.

[0020] Fig.12 is a flow chart illustrating an exemplary method of registering another virtual device based on another voice command.

[0021] Fig.13 An example of functional components executed by a processor of an MR device is shown.

[0022] Fig.14 is a block diagram of an electronic device in a network environment according to various embodiments. DETAILED DESCRIPTION

[0023] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that a person of ordinary skill in the art to which the present disclosure belongs can easily implement the present disclosure. However, the present disclosure can be implemented in several different forms and is not limited to the embodiments described herein. With respect to the description of the accompanying drawings, the same or similar reference numerals may be used for the same or similar components. In addition, in the accompanying drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and brevity.

[0024] Figure 1 An example of an environment including an exemplary electronic device and a mixed reality (MR) device is shown.

[0025] refer to Figure 1 , the environment 100 may include an electronic device 101 , an MR device 102 (or an extended reality (XR) device 102 ), another electronic device 103 , and an external electronic device 104 .

[0026] For example, the electronic device 101 may store a software application for controlling or managing another device through the electronic device 101. For example, the electronic device 101 may use the software application to control or manage real devices (or physical devices) such as the MR device 102 and another electronic device 103 and virtual devices related to the MR device 102. For example, the electronic device 101 may control another device by executing the software application based on the user input received for the user interface of the software application. For example, the software application may be used to register another device based on the user input received when the user interface is displayed. For example, the software application may be used to change the state of another device based on the user input received when the user interface is displayed. For example, the user input may include a touch input for the user interface displayed on the display of the electronic device 101 and a voice signal obtained by the microphone of the electronic device 101 when the user interface is displayed. For example, when the user interface is focused (focus), the voice signal may be recognized for the function provided by the software application. For example, the user interface being focused may indicate that the pointer (or cursor) is positioned relative to the user interface. For example, the user interface being focused may indicate that the user interface is in a state capable of receiving input (or user input). However, it is not limited thereto.

[0027] For example, the MR device 102 may include a device for providing an augmented reality (AR) service, a mixed reality (MR) service, or an extended reality (XR) service. For example, the MR device 102 may include a display for providing an AR service, an MR service, or an XR service. For example, when the MR device 102 is an AR glasses, the display of the MR device 102 may be transparent. For example, when the MR device 102 is a video see-through (or visual) (VST) device, the display of the MR device 102 may be opaque.

[0028] For example, the MR device 102 can provide AR services, MR services, or XR services by displaying the real environment around the MR device 102 or an image representing the real environment and a virtual object on the display of the MR device 102. For example, when the MR device 102 is an AR glasses, the MR device 102 can display a virtual object in the real environment shown by the display of the MR device 102. For example, when the MR device 102 is a VST device, the MR device 102 can display a virtual object on an image obtained by a camera of the MR device 102. For example, the virtual object can be recognized by the user wearing the MR device 102 as a real object existing in the real environment. For example, the virtual object can include a virtual device 105. For example, the virtual device 105 can include a display. For example, the virtual device 105 can display an image on the display of the virtual device 105, such as a real device including a display. For example, since the MR device 102 displays the virtual device 105, the MR device 102 can display an image on the display of the virtual device 105. For example, an image may be displayed on a display of the virtual device 105 based on data received from the electronic device 101 to the MR device 102 via the connection 112. For example, the connection 112 between the electronic device 101 and the MR device 102 may include connecting the electronic device 101 directly to the MR device 102 and / or connecting the electronic device 101 to the MR device 102 via at least one intermediate node. For example, an image may be displayed on a display of the virtual device 105 based on data received from another electronic device 103 to the MR device 102 via the connection 123. For example, the connection 123 between the MR device 102 and another electronic device 103 may include connecting the MR device 102 directly to the another electronic device 103 and / or connecting the MR device 102 to the another electronic device 103 via at least one intermediate node. For example, an image may be displayed on a display of the virtual device 105 based on data received from the MR device 102 via the electronic device 101 from another electronic device 103 using the connection 112 and the connection 113. For example, the connection 113 between the electronic device 101 and the other electronic device 103 may include directly connecting the electronic device 101 to the other electronic device 103 and / or connecting the electronic device 101 to the other electronic device 103 through at least one intermediate node.

[0029] For example, the software application may be used to display an image on a display of the virtual device 105. For example, a user input for displaying an image on a display of the virtual device 105 may be received by the software application. For example, the software application may be executed by the electronic device 101, the MR device 102, and / or another electronic device 103.

[0030] For example, the other electronic device 103 may be a device that provides data for displaying an image on a display of the virtual device 105. For example, a software application may be used to provide the data. For example, a user input for providing the data may be received through the software application. For example, the software application may be executed by the electronic device 101, the MR device 102, and / or the other electronic device 103.

[0031] For example, the external electronic device 104 may include a device that uses a software application to provide services for managing or controlling the electronic device 101, the MR device 102, and / or another electronic device 103 within the external electronic device 104, and may be a server. For example, the external electronic device 104 may support at least a portion of the operation of the electronic device 101, the MR device 102, and / or another electronic device 103 for displaying an image on a display of the virtual device 105 through the software application. For example, at least a portion of the support may be performed through a connection 114, a connection 124, and / or a connection 134. For example, the connection 114 between the electronic device 101 and the external electronic device 104 may include directly connecting the electronic device 101 to the external electronic device 104 and / or connecting the electronic device 101 to the external electronic device 104 through at least one intermediate node. For example, the connection 124 between the MR device 102 and the external electronic device 104 may include directly connecting the MR device 102 to the external electronic device 104 and / or connecting the MR device 102 to the external electronic device 104 through at least one intermediate node. However, it is not limited thereto.

[0032] The following description may provide various example operations for displaying an image on a display of a virtual device 105 displayed on a display of the MR device 102 via the electronic device 101 , the MR device 102 , another electronic device 103 , and / or an external electronic device 104 within the environment 100 .

[0033] Figure 2 is a simplified block diagram of an exemplary electronic device. The electronic device may be Figure 1 Electronic device 101.

[0034] refer to Figure 2, the electronic device 101 may include a processor (eg, including a processing circuit) 210 , a display 220 , a microphone 230 , a communication circuit 240 , and a memory 250 .

[0035] For example, processor 210 may include various processing circuits, including, for example Fig.14 For example, the display 220 may include at least a portion of the processor 1420. Fig.14 For example, the microphone 230 may include Fig.14 For example, the communication circuit 240 may include at least a portion of the input module 1450. Fig.14 For example, the memory 250 may include at least a portion of the communication module 1490 (or at least a portion of the wireless communication module 1492). Fig.14 at least a portion of memory 1430.

[0036] For example, the processor 210 may be operably coupled to each of the display 220, the microphone 230, the communication circuit 240, and the memory 250. For example, the processor 210 operably coupled to each of the display 220, the microphone 230, the communication circuit 240, and the memory 250 may instruct the processor 210 to be directly connected to each of the display 220, the microphone 230, the communication circuit 240, and the memory 250. For example, the processor 210 operably coupled to each of the display 220, the microphone 230, the communication circuit 240, and the memory 250 may instruct the processor 210 to be connected to each of the display 220, the microphone 230, the communication circuit 240, and the memory 250 through another component of the electronic device 101. For example, the processor 210 operably coupled to each of the display 220, the microphone 230, the communication circuit 240, and the memory 250 may instruct each of the display 220, the microphone 230, the communication circuit 240, and the memory 250 to operate based on the instructions executed by the processor 210. For example, the processor 210 operatively coupled to each of the display 220, the microphone 230, the communication circuit 240, and the memory 250 may instruct each of the display 220, the microphone 230, the communication circuit 240, and the memory 250 to be controlled by the processor 210. However, it is not limited thereto.

[0037] For example, the processor 210 may be configured to perform at least a portion of the operations to be shown below.

[0038] For example, Figure 2 The components shown in FIG. 1 may be included in the MR device 102 .

[0039] In the following example, when the MR device 102 displays an image through the display of the virtual device 105 without the assistance of the electronic device 101, at least a part of the operation of the electronic device 101 may be replaced with the operation of the MR device 102. In this case, the operations of the electronic device 101 and the MR device 102 performed based on the communication between the electronic device 101 and the MR device 102 may be omitted. In this case, the operation of the electronic device 101 performed based on the communication between the electronic device 101 and another electronic device 103 may be replaced with the operation of the MR device 102 performed based on the communication between the MR device 102 and another electronic device 103. In this case, the operation of the electronic device 101 performed based on the communication between the electronic device 101 and the external electronic device 104 may be replaced with the operation of the MR device 102 performed based on the communication between the MR device 102 and the external electronic device 104.

[0040] Figure 3 is a flow chart showing an exemplary method of registering a virtual device. The method may be Figure 1 Electronic devices 101, Figure 2 Electronic equipment 101 or Figure 2 The processor 210 of the embodiment of the present invention is executed. In the following example embodiments, each operation can be performed sequentially, but not necessarily sequentially. For example, the order of each operation can be changed, and at least two operations can be performed in parallel.

[0041] refer to Figure 3 In operation 301, the processor 210 may execute a software application for managing or controlling another device through the electronic device 101. For example, the software application may be executed in a foreground state, where the foreground state is displayed on a display (e.g., Figure 2 The user interface of the software application may be displayed on the display 220 in the display 220. For example, the software application may be executed in a background state, in which the user interface is not displayed on the display 220. For example, the state of the software application may change from a foreground state to a background state, or from a background state to a foreground state.

[0042] In operation 303, the processor 210 may register the virtual device using a software application (eg, Figure 1 For example, the virtual device 105 may be a device included in the virtual space or the MR space. For example, the virtual device 105 may be displayed in the virtual space or the MR space accessible MR device (e.g., Figure 1 For example, in a registered virtual space or a device accessible to the MR space (e.g., Figure 1Under the condition that the MR device 102 is not included, the registration of the virtual device 105 can be performed. However, this is not limited to this.

[0043] For example, the processor 210 may register the device information of the virtual device 105 and the information about the location of the virtual device 105 as being associated with the virtual device 105 registered using the software application. For example, while registering the virtual device 105 using the software application, the device information and the information about the location of the virtual device 105 may be input based on a touch input received through the user interface or based on a voice signal obtained through the microphone 230.

[0044] Can be Figure 4a 2 shows a virtual device 105 registered using a software application to be associated with device information and information about the location of the virtual device 105 .

[0045] Figure 4a An exemplary method of registering a virtual device by using a software application for controlling another device is shown.

[0046] refer to Figure 4a , as in state 400, a processor (e.g., Figure 2 The processor 210 of the software application may display a user interface 410 of the software application. For example, the user interface 410 may include a user interface for registering a virtual device (e.g., Figure 1 For example, the processor 210 may receive a touch input 412 for the executable object 411. For example, the processor 210 may identify the MR device (e.g., Figure 1 The MR device 102 of FIG. 1 is worn in response to the touch input 412 .

[0047] For example, the processor 210 may recognize whether the MR device 102 is worn based on a signal indicating the state of the MR device 102 received from the MR device 102. For example, the signal may include information indicating movement or a change in movement of the MR device 102 obtained by a sensor of the MR device 102. For example, the signal may include information indicating whether the MR device 102 is adjacent to an external object (e.g., a part of the user's body (e.g., the head)) or information indicating the distance between the MR device 102 and the external object obtained by a sensor of the MR device 102. For example, the processor 210 may recognize that the MR device 102 is worn or that the MR device 102 is not worn based on the signal.

[0048] For example, the processor 210 may request a signal from the MR device 102 without receiving a signal indicating a state of the MR device 102 from the MR device 102. For example, the MR device 102 may receive a request for a signal. In response to the request, the MR device 102 may obtain a signal based on data obtained by a sensor of the MR device 102. The MR device 102 may send the signal to an electronic device (e.g., Figure 1 The processor 210 may recognize, based on a signal received from the MR device 102, that the MR device 102 is worn or that the MR device 102 is not worn.

[0049] For example, in response to recognizing that the MR device 102 is worn, the processor 210 may transmit a signal for registration of the virtual device 105 to the MR device 102. For example, the signal for registration may be transmitted to recognize device information and the location of the virtual device 105 using the MR device 102. For example, the MR device 102 may receive a signal for registration from the electronic device 101.

[0050] For example, as in state 420, in response to a signal for registration, the MR device 102 may display a visual object 430 representing the virtual device 105 to be registered by the software application and the environment around the MR device 102, or on an image representing the environment around the MR device 102. For example, the visual object 430 may be moved based on a user input recognized by the MR device 102 (e.g., gaze input, gesture input through movement of a controller connected to the MR device 102, and / or voice input). For example, the size of the visual object 430 may be changed based on a user input recognized by the MR device 102 (e.g., gaze input, gesture input through movement of a controller connected to the MR device 102, and / or voice input). For example, the shape of the visual object 430 may be changed based on a user input recognized by the MR device 102 (e.g., gaze input, gesture input through movement of a controller connected to the MR device 102, and / or voice input).

[0051] For example, in state 420, MR device 102 may receive user input 440 indicating that visual object 430 moves to area 435 and user input 445 indicating that the size of visual object 430 increases. For example, MR device 102 may change state 420 to state 450 based on user input 440 and user input 445.

[0052] For example, in state 450, the MR device 102 may display a visual object 430 having a size identified based on the user input 445 in a position identified based on the user input 440. For example, the MR device 102 may receive device information of the MR device 102 and a user input (e.g., gaze input, gesture input through movement of a controller connected to the MR device 102, and / or voice input) indicating determination of the position of the MR device 102 while displaying the visual object 430. For example, in response to the user input, the MR device 102 may obtain first information indicating the position of the visual object 430 as the position of the virtual device 105 and obtain second information indicating the size of the visual object 430 as the size of the virtual device 105. For example, the MR device 102 may send the first information and the second information to the electronic device 101. For example, the processor 210 may receive the first information and the second information from the MR device 102.

[0053] For example, the processor 210 may register the virtual device 105 based at least in part on receiving the first information and the second information. For example, the processor 210 may send information for registering the virtual device 105 to an external electronic device (e.g., Figure 1 For example, the virtual device 105 may be registered in the external electronic device 104 based on the information. For example, the virtual device 105 registered in the external electronic device 104 may be associated with the location of the virtual device 105 and the device information of the virtual device 105.

[0054] For example, the processor 210 may identify the authority to access the virtual device 105 after completing the registration of the virtual device 105 or after receiving the first information and the second information. For example, the processor 210 may associate the virtual device 105 with information about another user account using the software application based on a user input indicating authorization to access the virtual device 105 through another user account (e.g., a user account of a family member) different from the user account used within the electronic device 101. For example, the processor 210 may send a signal to the external electronic device 104 indicating that the other user account is associated with the virtual device 105.

[0055] For example, the processor 210 may display an executable object corresponding to the virtual device 105 in the user interface 410 after the registration of the virtual device 105 is completed. The executable object in the user interface 410 may be displayed by Figure 4b Shown.

[0056] Figure 4b An example of a user interface including a visual object corresponding to a virtual device registered by using a software application for controlling another device is shown.

[0057] refer to Figure 4b As in state 460, user interface 410 may include executable objects 470. For example, executable objects 470 may include executable object 465 corresponding to virtual device 105. For example, executable objects 470 may include executable object 466 corresponding to another electronic device 103 as a real device, executable object 467 corresponding to a real device, and executable object 468 corresponding to another virtual device. For example, in user interface 410, executable objects 470 may be categorized. For example, executable objects 470 may be categorized based on location, such as Figure 4b For example, in user interface 410, executable objects 465 and 466 may indicate that virtual device 105 and another electronic device 103 are located in a master bedroom, and executable objects 467 and 468 may indicate that a real device corresponding to executable object 467 and another virtual device corresponding to executable object 468 are located in a living room.

[0058] For example, each of executable objects 470 may include a quick access element for controlling a device corresponding to each of executable objects 470. For example, executable object 465 may include quick access element 471. Although not shown in FIG. Figure 4b , the processor 210 may display a user interface 410 including a plurality of executable objects for controlling or managing a device corresponding to each of the executable objects 470 in response to a touch input on another area of ​​each of the executable objects 470 that is different from the area of ​​the quick access element. Figure 4b , but the processor 210 may control each of the executable objects 470 based on a function corresponding to the quick access element in response to a touch input on an area of ​​the quick access element.

[0059] Return to reference Figure 4a , the processor 210 may provide a notification or an alarm for notifying or guiding the MR device 102 to be worn in response to recognizing that the MR device 102 is not worn. For example, the notification or the alarm may be provided from the electronic device 101 or from the MR device 102. For example, when the notification or the alarm is provided from the electronic device 101, the notification or the alarm may be displayed through the display 220, output through the speaker of the electronic device 101, or output as a vibration through the actuator of the electronic device 101. For example, when the notification or the alarm is provided from the MR device 102, the notification or the alarm may be output through the speaker of the MR device 102. However, it is not limited thereto.

[0060] For example, processor 210 may register virtual device 105 based on user input received through user interface 410 in response to identifying MR device 102 is not worn. For example, registration of virtual device 105 by electronic device 101 may be at least partially different from registration of virtual device 105 by MR device 102.

[0061] Although not in Figure 4a , but the processor 210 may register the virtual device 105 by recognizing a user input for inputting text (or characters) indicating device information and the location of the virtual device 105 through the user interface 410 .

[0062] For example, information of a real device (e.g., information of a real product) may be used for registration of the virtual device 105. For example, the processor 210 may identify device information of the virtual device 105 based on the information about the real device by recognizing a user input for inputting information about a real device having a type corresponding to the type of the virtual device 105. The processor 210 may register the virtual device 105 based on the recognition.

[0063] As described above, the electronic device 101 may use a software application to register the virtual device 105 displayed on the display of the MR device 102. For example, the registration of the virtual device 105 initiated from the electronic device 101 may be partially processed by the MR device 102. For example, the electronic device 101 may perform the registration of the virtual device 105 based on a user experience that may be provided by the MR device 102 and a user experience that may be provided by the electronic device 101.

[0064] Figure 5 is a flow chart showing an exemplary method of displaying an image through a display of a virtual device. The method may be performed by Figure 1 Electronic devices 101, Figure 2 Electronic equipment 101 or Figure 2 The processor 210 of the embodiment of the present invention is executed. In the following example embodiments, each operation can be performed sequentially, but not necessarily sequentially. For example, the order of each operation can be changed, and at least two operations can be performed in parallel.

[0065] refer to Figure 5 , in operation 501, Figure 2 The processor 210 may be based on the microphone (eg, Figure 2 The invention also provides a method for identifying a voice signal obtained by a microphone 230 of the virtual device (e.g., Figure 1The display of a virtual device 105) is registered by using a software application for controlling another device through the electronic device (101). For example, a voice command may directly (or explicitly) or indirectly (or implicitly) instruct to perform a display through the display of the virtual device 105. Figure 6 , Figure 7 , Figure 8 and Fig.10 Recognition of a voice command based on recognition of a voice signal is shown.

[0066] In operation 503, the processor 210 may identify an MR device (eg, Figure 1 Whether the MR device 102 is worn.

[0067] For example, since the voice command indicates a display through the display of the virtual device 105 that can be displayed through the MR device 102 , the processor 210 may recognize whether the MR device 102 is worn in response to the voice command.

[0068] For example, the processor 210 may communicate with the user through a communication circuit (e.g., Figure 2 The processor 210 may receive a signal indicating the state of the MR device 102 from the MR device 102 connected to the electronic device 101 (the communication circuit 240). For example, the processor 210 may recognize whether the MR device 102 is worn based on the signal. For example, the signal indicating the state of the MR device 102 may include information indicating the movement (and / or posture) of the MR device 102 obtained by a sensor (e.g., an acceleration sensor and / or a gyroscope sensor) of the MR device 102. For example, the signal indicating the state of the MR device 102 may include information indicating the distance between the MR device 102 and an external object or information indicating whether an external object is adjacent to the MR device 102 obtained by a sensor (e.g., a proximity sensor and / or a UWB communication circuit) of the MR device 102.

[0069] For example, the processor 210 may perform operation 505 based on recognizing that the MR device 102 is worn, and may perform operation 509 based on recognizing that the MR device 102 is not worn.

[0070] In operation 505, under the condition that the MR device 102 is worn, the processor 210 may obtain an image to be displayed on the display of the virtual device 105 displayed by the MR device 102 based on the device information registered as associated with the virtual device 105 using the software application. For example, the processor 210 may perform operation 505 based on the device information so as to obtain an image suitable for display of the virtual device 105.

[0071] For example, it is possible to obtain a signal from an external electronic device (e.g., Figure 1For example, the processor 210 may request the external electronic device 104 to send the device information of the virtual device 105 in response to recognizing that the MR device 102 is worn. The external electronic device 104 may send the device information to the electronic device 101 in response to the request. The electronic device 101 may obtain an image based on the device information.

[0072] For example, the device information may be stored in the electronic device 101 in association with the virtual device 105. For example, the processor 210 may obtain the device information by identifying information associated with the virtual device 105 in response to identifying that the MR device 102 is worn. For example, the processor 210 may obtain an image based on the device information.

[0073] For example, by Figures 3 to 4b The device information described in the description registered as being associated with the virtual device 105 may include data regarding the size of the display of the virtual device 105 and / or data regarding the aspect ratio of the display of the virtual device 105 .

[0074] In operation 507, the processor 210 may transmit a signal for displaying an image on the display of the virtual device 105 displayed by the MR device 102 to the MR device 102 through the communication circuit 240. For example, the signal may cause the MR device 102 to display an image on the display of the virtual device 105 displayed by the display of the MR device 102. For example, the virtual device 105 located in the virtual space or the MR space provided by the MR device 102 may display an image on the display of the virtual device 105. For example, the display may be controlled by the MR device 102 based on the signal.

[0075] For example, the images may include still images, dynamic images, execution screens of software applications, wallpapers, and lock screens.

[0076] In operation 509 , the processor 210 may provide a notification notifying that the MR device 102 is to be worn under the condition that the MR device 102 is not worn.

[0077] For example, the notification may be provided by a component of the electronic device 101. For example, the notification may be provided by a display of the electronic device 101 (e.g., Figure 2 The notification may be displayed by the display 220 of the electronic device 101. For example, the notification may be output through a speaker of the electronic device 101. For example, the notification may be output as a vibration through an actuator of the electronic device 101. However, this is not limited thereto.

[0078] For example, the notification may be provided through a component of the MR device 102. For example, the notification may be output through a speaker of the MR device 102. However, this is not limiting.

[0079] As described above, the electronic device 101 can receive the signal from the microphone (e.g. Figure 2 The electronic device 101 extracts a voice command indicating the virtual device 105 from a voice signal received by a microphone 230 of the electronic device 101, and recognizes the state of the MR device 102 based on the extracted voice command to control the virtual device 105. For example, the electronic device 101 can communicate with the MR device 102 based on recognizing that the MR device 102 is in a state that can be used to control the virtual device 105 to control the virtual device 105 through the MR device 102. For example, the electronic device 101 can add a service provided by the MR device 102 to a service provided by the electronic device 101 based on these communications. For example, the electronic device 101 can provide an enhanced service or an enhanced user experience.

[0080] Figure 6 is a flow chart showing an exemplary method for recognizing a voice command based on a display position. The method may be Figure 1 Electronic devices 101, Figure 2 Electronic equipment 101 or Figure 2 The processor 210 of the embodiment of the present invention is executed. In the following example embodiments, each operation can be performed sequentially, but not necessarily sequentially. For example, the order of each operation can be changed, and at least two operations can be performed in parallel.

[0081] Figure 6 Operation 601 and operation 603 may include Figure 5 However, this is not limited to this. For example, operation 601 and operation 603 may be independent of Figure 5 Operation 501.

[0082] refer to Figure 6 , in operation 601, Figure 2 The processor 210 may be based on the microphone (eg, Figure 2 The display position can be identified by recognizing a voice signal obtained by microphone 230 of the processor 210. For example, the processor 210 can obtain a voice signal corresponding to "display in the master bedroom" through microphone 230. For example, the processor 210 can identify the display position indicated by the voice signal as "master bedroom" based on the recognition of the voice signal.

[0083] In operation 603, the processor 210 may, based on identifying that the display location corresponds to a virtual device (e.g., Figure 1The processor 210 may identify the position of each device registered with the software application based on the display position. For example, as a result of the identification, the processor 210 may identify, from at least a portion of the voice signal, a voice command indicating display through the display of the virtual device 105 by identifying that the position of the virtual device 105 corresponds to the display position.

[0084] As described above, even when the voice signal does not include a portion indicating the virtual device 105, the electronic device 101 can use the location associated with each device registered using the software application to recognize at least a portion of the voice signal as a voice command indicating display through the display of the virtual device 105.

[0085] Figure 7 is a flowchart illustrating an exemplary method of obtaining an image to be displayed by a display of a virtual device based on information obtained from another electronic device. The method may be Figure 1 Electronic devices 101, Figure 2 Electronic equipment 101 or Figure 2 The processor 210 of the embodiment of the present invention is executed. In the following example embodiments, each operation can be performed sequentially, but not necessarily sequentially. For example, the order of each operation can be changed, and at least two operations can be performed in parallel.

[0086] Figure 7 At least a portion of operations 701 to 709 may be performed with Figure 5 Operation 501, Figure 5 Operation 503, Figure 5 Operation 505, Figure 6 Operation 601 and Figure 6 is related to at least a portion of operation 603 .

[0087] refer to Figure 7 In operation 701, Figure 2 The processor 210 may be based on the microphone (eg, Figure 2 The voice command is further indicated by the recognition of the voice signal obtained by the microphone 230 of the electronic device (e.g., Figure 1 Another electronic device 103) obtains the information to be transmitted on the virtual device (for example, Figure 1 Data of an image displayed on a display of a virtual device 105).

[0088] For example, the processor 210 may obtain a voice signal corresponding to "displaying the screen of the living room TV in the master bedroom" through the microphone 230. For example, the processor 210 may recognize, based on the recognition of the voice signal, that the data of the image to be displayed on the display of the virtual device 105 is obtained from another electronic device 103 corresponding to the "living room TV".

[0089] In operation 703, the processor 210 may identify the MR device (eg, Figure 1 For example, in operation 701, the processor 210 may identify whether the MR device 102 is worn in response to the recognition of the voice command. For example, the processor 210 may identify that the MR device 102 is worn based on a signal received from the MR device 102.

[0090] In operation 705, the processor 210 may request data from another electronic device 103 through the communication circuit 240 based on the recognized voice command in response to recognizing that the MR device 102 is worn. For example, the processor 210 may send a request based on information for accessing another electronic device 103 registered with the software application. The other electronic device 103 may receive the request.

[0091] In operation 707 , the processor 210 may receive data of an image from another electronic device 103 .

[0092] In operation 709 , the processor 210 may obtain an image to be displayed on a display of the virtual device 105 based on the data and device information of the virtual device 105 .

[0093] For example, the image may correspond to another image displayed by the display of another electronic device 103. For example, an image displayed on the display of another electronic device 103 may be mirrored on the display of the virtual device 105. For example, the image may be a mirror image of another image. For example, the content represented by the image may be substantially the same as the content represented by the other image. For example, the arrangement of visual objects in the image may be the same as or different from the arrangement of visual objects in another image. For example, when the size of the display of the virtual device 105 is different from the size of the display of the other electronic device 103, or the aspect ratio of the display of the virtual device 105 is different from the aspect ratio of the display of the other electronic device 103, the arrangement of visual objects in the image may be different from the arrangement of visual objects in another image. However, it is not limited thereto.

[0094] For example, the image may be an image extending from another image displayed by a display of another electronic device 103. For example, the image and the other image may provide a single screen. For example, the image may represent a portion of the single screen, and the other image may represent another portion of the single screen.

[0095] As described above, the electronic device 101 may obtain or convert an image provided from another electronic device 103, and provide the image to the MR device 102 for display on the display of the virtual device 105. For example, through these operations, the electronic device 101 may display an image from another electronic device 103 on the display of the virtual device 105 displayed by the MR device 102. For example, the electronic device 101 may use a software application to deliver a service provided by another electronic device 103 to the MR device 102. For example, since the capabilities of the MR device 102 are different from those of the another electronic device 103, the electronic device 101 may provide a service that expands the capabilities of the another electronic device 103.

[0096] Figure 8 is a flow chart illustrating an exemplary method for recognizing a voice command based on a location of an electronic device, a location of another electronic device, and a location of a virtual device. The method may be Figure 1 Electronic devices 101, Figure 2 Electronic equipment 101 or Figure 2 The processor 210 of the embodiment of the present invention is executed. In the following example embodiments, each operation can be performed sequentially, but not necessarily sequentially. For example, the order of each operation can be changed, and at least two operations can be performed in parallel.

[0097] Figure 8 Operations 801 to 805 may include Figure 5 However, this is not limited to this. For example, operations 801 to 805 may be independent of Figure 5 Operation 501.

[0098] refer to Figure 8 , in operation 801, Figure 2 The processor 210 may be based on the microphone (eg, Figure 2 The recognition of the voice signal obtained by the microphone 230) and the recognition of the voice signal indicates that the display of the display of another electronic device 103 registered by using the software application and the virtual device (e.g., Figure 1 For example, the processor 210 may obtain a voice signal corresponding to "turn on the TV" through the microphone 230. For example, the processor 210 may use a software application to recognize the "TV" indicated by the voice signal based on the recognition of the voice signal. For example, the processor 210 may recognize that the TV registered using the software application is another electronic device (e.g., Figure 1 another electronic device 103) and the virtual device 105 to recognize that the voice signal indicates the display through the display of the other electronic device 103 and the display through the display of the virtual device 105.

[0099] In operation 803, the processor 210 may recognize that the position of the electronic device 101 corresponds to the position of the virtual device 105 among the position of the other electronic device 103 and the position of the virtual device 105. For example, in response to the recognition, the processor 210 may use a communication circuit (e.g., Figure 2 The processor 210 may identify the location of the electronic device 101 using the communication circuit 240 of the electronic device 101. For example, in response to the identification, the processor 210 may identify the location of another electronic device 103 registered using the software application. For example, in response to the identification, the processor 210 may identify the location of the virtual device 105 registered using the software application. For example, the processor 210 may identify that the location of the electronic device 101 corresponds to the location of the virtual device 105 among the location of the another electronic device 103 and the location of the virtual device 105.

[0100] In operation 805, the processor 210 may recognize a voice command instructing display through the display of the virtual device 105 based on recognizing that the location of the electronic device 101 corresponds to the location of the virtual device 105. For example, since the probability that information about the location of the service is included in the voice signal from the user can be reduced when the user's intention is to receive the service at his own location, the processor 210 may recognize the voice command based on recognizing that the location corresponds to the location of the virtual device 105.

[0101] As described above, even when the voice signal does not include a portion indicating the virtual device 105, the electronic device 101 can use the location associated with each device registered using the software application to recognize at least a portion of the voice signal as a voice command indicating display through the display of the virtual device 105.

[0102] Fig. 9 is a flowchart showing an exemplary method for obtaining an image based on device information of a virtual device sent to an external electronic device. The method may be Figure 1 Electronic devices 101, Figure 2 Electronic equipment 101 or Figure 2 The processor 210 of the embodiment of the present invention is executed. In the following example embodiments, each operation can be performed sequentially, but not necessarily sequentially. For example, the order of each operation can be changed, and at least two operations can be performed in parallel.

[0103] Fig. 9 At least a portion of operations 901 to 911 may be performed with Figure 5 The operations 501, 503 and 505 are related to at least a portion of the operations 501, 503 and 505.

[0104] refer to Fig. 9 , in operation 901, Figure 2The processor 210 may display a display (eg, Figure 2 For example, another image corresponding to the image to be obtained in operation 911 may be displayed on the display 220. For example, the first data may be data for displaying another image. For example, the first data may be transmitted to the display 220 via a communication circuit (e.g., Figure 2 The communication circuit 240 of the present invention receives first data from an external electronic device (eg, a server providing a streaming service).

[0105] In operation 903, the processor 210 may display another image on the display 220 while receiving the signal through a microphone (eg, Figure 2 The microphone 230 in the apparatus obtains the voice signal.

[0106] In operation 905, the processor 210 may identify an indication virtual device (eg, Figure 1 For example, operation 905 may correspond to: Figure 5 Operation 501.

[0107] In operation 907, the processor 210 may identify the MR device (eg, Figure 1 MR device 102) is worn. For example, operation 907 may correspond to Figure 5 Part of operation 503.

[0108] In operation 909, the processor 210 may, in response to recognizing that the MR device 102 is worn, communicate with the MR device 102 through the communication circuit (eg, Figure 2 The processor 210 may transmit the device information of the virtual device 105 to the external electronic device through the communication circuit 240 of the virtual device 105. For example, the processor 210 may transmit the device information to provide the content provided from the external electronic device on the display of the virtual device 105. For example, the processor 210 may transmit the device information so that the external electronic device sets an image to be displayed on the display of the virtual device 105.

[0109] For example, the virtual device 105 may not be recognized or identified by the external electronic device. For example, the virtual device 105 may be transparent to the external electronic device. However, it is not limited to this. For example, the processor 210 may send the identification information of the virtual device 105 together with the device information to the external electronic device so that the external electronic device can recognize or identify the virtual device 105.

[0110] In operation 911, the processor 210 may receive second data obtained based on device information from an external electronic device through the communication circuit 240 to obtain an image. For example, the image may correspond to another image. For example, the content provided by the image may correspond to the content provided by another image. For example, the arrangement of visual objects in the image may be the same as or different from the arrangement of visual objects in another image. For example, when the size of the display of the virtual device 105 is different from the size of the display 220, or the aspect ratio of the display of the virtual device 105 is different from the aspect ratio of the display 220, the arrangement of visual objects in the image may be different from the arrangement of visual objects in another image. However, it is not limited to this.

[0111] As described above, the electronic device 101 can display content from an external electronic device connected to the electronic device 101 on the display of the virtual device 105. For example, the electronic device 101 can reduce the amount of resources consumed to obtain an image to be displayed on the display of the virtual device 105 by requesting the configuration of the image to be displayed on the display of the virtual device 105 from the external electronic device. For example, the resources can indicate calculations performed for the image to be displayed on the display of the virtual device 105. However, this is not limited thereto.

[0112] Fig.10 is a flow chart illustrating an exemplary method for recognizing a voice command based on the orientation of a display of another electronic device and the orientation of a display of a virtual device. The method may be Figure 1 Electronic devices 101, Figure 2 Electronic equipment 101 or Figure 2 The processor 210 of the embodiment of the present invention is executed. In the following example embodiments, each operation can be performed sequentially, but not necessarily sequentially. For example, the order of each operation can be changed, and at least two operations can be performed in parallel.

[0113] Fig.10 At least a portion of operations 1001 to 1007 may include Figure 5 However, this is not limited to this. For example, operations 1001 to 1007 may be independent of Figure 5 Operation 501.

[0114] refer to Fig.10 , in operation 1001, Figure 2 The processor 210 can recognize the Figure 2 The voice signal obtained by the microphone 230) indicates that the voice signal is transmitted through another electronic device (for example, Figure 1 The display of another electronic device 103) and the display of the virtual device (for example, Figure 1For example, the other electronic device 103 may be a real device registered using a software application. For example, the voice signal may correspond to "turn on the TV". For example, the processor 210 may identify the other electronic device 103 and the virtual device 105 based on recognizing that the voice signal indicates "TV", and the virtual device 105 is a device including a display among the devices registered by the software application. For example, the processor 210 may recognize that the voice signal indicates display through the display of the other electronic device 103 and display through the display of the virtual device 105.

[0115] In operation 1003, the processor 210 may recognize that the location of the other electronic device 103 and the location of the virtual device 105 are included in one area. For example, the processor 210 may recognize the location of the other electronic device 103 registered using the software application as being associated with the other electronic device 103. For example, the processor 210 may recognize the location of the virtual device 105 registered using the software application as being associated with the virtual device 105. For example, the processor 210 may recognize that the location of the other electronic device 103 and the location of the virtual device 105 are located in one area (e.g., “master bedroom”).

[0116] In operation 1005, the processor 210 may identify the direction of the display of the other electronic device 103 and the direction of the display of the virtual device 105. For example, the fact that the other electronic device 103 and the virtual device 105 are within a single area indicates that the device corresponding to the user's intention cannot be identified among the other electronic device 103 and the virtual device 105 based on the position, and the processor 210 may identify the direction of the display of the other electronic device 103 and the direction of the display of the virtual device 105. For example, the processor 210 may identify the direction of the display of the other electronic device 103 and the direction of the display of the virtual device 105 based on the position of the other electronic device 103 within the single area and the position of the virtual device 105 within the single area.

[0117] In operation 1007, the processor 210 may recognize a voice command indicating a display on the display of the virtual device 105 based on the direction of the display of the other electronic device 103 and the direction of the display of the virtual device 105. For example, the processor 210 may use a communication circuit (e.g., Figure 2The processor 210 may recognize the position of the electronic device 101 in a single area by using the communication circuit 240 of the electronic device 101. For example, the processor 210 may recognize the voice command based on the position of the electronic device 101, the direction of the display of the other electronic device 103, and the direction of the display of the virtual device 105. For example, the processor 210 may recognize the voice command by recognizing that the electronic device 101 is located in front of the display of the virtual device 105 among the display of the other electronic device 103 and the display of the virtual device 105 based on the position of the electronic device 101, the direction of the display of the other electronic device 103, and the direction of the display of the virtual device 105.

[0118] Fig.10 An example of identifying one device based on the direction of the display when the voice signal indicates two or more devices is shown, but this is for convenience of explanation. The electronic device 101 may identify one of the two or more devices based on various other criteria as well as the direction of the display.

[0119] For example, the processor 210 may identify device information of the other electronic device 103 and device information of the virtual device 105 based on identifying the other electronic device 103 and the virtual device 105 from the voice signal.

[0120] For example, the processor 210 may identify the size of the display of the other electronic device 103 from the device information of the other electronic device 103, and may identify the size of the display of the virtual device 105 from the device information of the virtual device 105. For example, the processor 210 may compare the size of the display of the other electronic device 103 with the size of the display of the virtual device 105. For example, the processor 210 may identify a voice command instructing display through the display of the virtual device 105 based on the fact that the size of the display of the virtual device 105 is larger than the size of the display of the other electronic device 103. However, it is not limited thereto.

[0121] For example, the processor 210 may identify the resolution of the display of the other electronic device 103 from the device information of the other electronic device 103, and may identify the resolution of the display of the virtual device 105 from the device information of the virtual device 105. For example, the processor 210 may compare the resolution of the display of the other electronic device 103 with the resolution of the display of the virtual device 105. For example, the processor 210 may identify a voice command instructing display through the display of the virtual device 105 based on the fact that the resolution of the display of the virtual device 105 is higher than the resolution of the display of the other electronic device 103. However, it is not limited thereto.

[0122] For example, the processor 210 may identify the mobility of the other electronic device 103 from the device information of the other electronic device 103, and identify the mobility of the virtual device 105 from the device information of the virtual device 105. For example, the processor 210 may also identify the movement of the electronic device 101 based on identifying the other electronic device 103 and the virtual device 105. For example, the movement may be identified by a sensor of the electronic device 101. For example, the processor 210 may identify that the mobility of the virtual device 105 is higher than the mobility of the other electronic device 103, and identify a voice command indicating the display through the display of the virtual device 105 based on identifying that the electronic device 101 is moving. However, it is not limited thereto.

[0123] As described above, even when the voice signal received through the microphone 230 does not explicitly indicate the virtual device 105, the electronic device 101 can recognize at least a portion of the voice signal as a voice command instructing display through the display of the virtual device 105 based on the device information of another electronic device 103 and the device information of the virtual device 105.

[0124] Fig.11 is a flow chart showing an exemplary method of controlling a virtual device located in a virtual space. The method may be Figure 1 Electronic devices 101, Figure 2 Electronic equipment 101 or Figure 2 The processor 210 of the embodiment of the present invention is executed. In the following example embodiments, each operation can be performed sequentially, but not necessarily sequentially. For example, the order of each operation can be changed, and at least two operations can be performed in parallel.

[0125] In operation 1101, when communicating with an MR device (e.g., Figure 1 When the connection using the software application of the MR device 102 is released, Figure 2 The processor 210 may identify a virtual device (eg, Figure 1 Another image displayed on the display of the virtual device 105).

[0126] For example, the virtual space may be a space different from the MR space. For example, unlike the MR space activated when accessed using the MR device 102, the virtual space may be maintained and activated even when the MR device 102 is not used to access the virtual space.

[0127] For example, another image may be provided to the software application from another software application different from the software application. For example, the other software application may be a software application for managing images obtained by a camera of the electronic device 101 or received to the electronic device 101. For example, the other software application may be a software application for managing or displaying a web page including another image. However, this is not limited thereto.

[0128] In operation 1103, the processor 210 may communicate with the user through a communication circuit (eg, Figure 2 The processor 210 may further include a communication circuit 240 (not shown) for transmitting data for displaying another image on the display of the virtual device 105 to an external electronic device that provides a virtual space, while releasing the connection with the MR device 102. For example, the operation 1103 may be performed using a software application. For example, even when the virtual space is not accessed using the MR device 102, the processor 210 may transmit data using a software application to set an image displayed by the display of the virtual device 105 in the virtual space.

[0129] As described above, while the connection with the MR device 102 is released, the electronic device 101 can control the virtual device 105 located in the virtual space accessible by the MR device 102 among the electronic device 101 and the MR device 102. The electronic device 101 can expand the usability of the virtual space by providing such a service.

[0130] Fig.12 is a flow chart showing an exemplary method for registering another virtual device based on another voice command. The method may be Figure 1 Electronic devices 101, Figure 2 Electronic equipment 101 or Figure 2 The processor 210 of the embodiment of the present invention is executed. In the following example embodiments, each operation can be performed sequentially, but not necessarily sequentially. For example, the order of each operation can be changed, and at least two operations can be performed in parallel.

[0131] In operation 1201, Figure 2 The processor 210 may be based on the microphone (eg, Figure 2 The method further comprises: identifying another voice signal obtained by the microphone 230 of the software application to identify another voice command, the other voice command indicating that the virtual device (for example, Figure 1 The processor 210 may display a display in another location different from the location of multiple devices of the virtual device 105). For example, the processor 210 may obtain a voice signal corresponding to "display an image in the kitchen" through the microphone 230. For example, the processor 210 may identify a device located in the "kitchen" as the location indicated by the voice signal among the devices registered using the software application. For example, the processor 210 may recognize another voice command based on the recognition.

[0132] In operation 1203, the processor 210 may display a user interface of a software application for registering another virtual device located at another location different from the virtual device 105 based on another voice command. For example, unlike a real device, since a virtual device can be positioned without restriction, the processor 210 may display a user interface for registering another virtual device. For example, the user interface may include text or visual objects for guiding the registration of another virtual device. For example, the user interface may be displayed by the user interface. Figures 3 to 4b The registration of the virtual device 105 shown provides the registration of another virtual device more simply. For example, the registration of another virtual device can be performed using the device information of the virtual device 105 registered with the software application before registering another virtual device. However, it is not limited to this.

[0133] As described above, the electronic device 101 can guide the registration of available virtual devices based on the recognition of the voice signal, regardless of the location. For example, the electronic device 101 can provide an enhanced user experience through guidance.

[0134] Fig.13 An example of functional components executed by a processor of an MR device is shown.

[0135] refer to Fig.13 , MR devices (e.g., Figure 1 The processor of the MR device 102 may use the input interface (e.g., including various circuits) 1310 to receive and recognize user input. For example, the user input may include line of sight input, gesture input through a controller, and / or voice input. For example, the processor of the MR device 102 may use the input interface 1310 to perform a function corresponding to the user input. Fig.13 Each module described may include, for example, various processing circuits and / or executable program instructions.

[0136] For example, when the user input is a voice input, the processor of the MR device 102 may recognize the user input using the voice recognition module 1320. For example, the processor of the MR device 102 may provide information about the user input recognized through the input interface 1310 to the voice recognition module 1320. For example, the processor of the MR device 102 may recognize the user input by performing natural language interpretation on the user input using the voice recognition module 1320.

[0137] For example, the processor of the MR device 102 may control another device through the MR device 102 using the software application 1330 as the software application shown above. For example, the processor of the MR device 102 may register the virtual device 105 and the other electronic device 103 using the software application 1330. For example, the processor of the MR device 102 may set an image (or content) to be displayed through the virtual device 105 and the other electronic device 103 using the software application 1330. For example, the software application 1330 may be executed based on information obtained from the external electronic device 104.

[0138] For example, the processor of the MR device 102 may use the output interface 1350 to control the image displayed on the display of the MR device 102 and the sound output through the speaker of the MR device 102. For example, the output interface 1350 may include a display controller (e.g., including various circuits) 1360, a rendering module 1370, and a sound controller (e.g., including various circuits) 1380. For example, the processor of the MR device 102 may use the display controller 1360 to decode an image for the virtual device 105 obtained from another device (e.g., the electronic device 101 and / or another electronic device 103). For example, the processor of the MR device 102 may encode an image displayed on the display of the virtual device 105 and provided to another device (e.g., the electronic device 101 and / or another electronic device 103) using the display controller 1360. However, it is not limited thereto.

[0139] For example, the processor of the MR device 102 may render an image in the MR space or virtual space using the rendering module 1370. For example, the processor of the MR device 102 may render an image obtained from the display controller 1360 using the rendering module 1370.

[0140] For example, the processor of the MR device 102 may control the sound output while displaying the image using the sound controller 1380. For example, the sound controller 1380 may perform encoding and / or decoding of the sound.

[0141] Fig.14 1 is a block diagram illustrating an electronic device 1401 in a network environment 1400 according to various embodiments. Fig.14, the electronic device 1401 in the network environment 1400 may communicate with the electronic device 1402 via the first network 1498 (e.g., a short-range wireless communication network), or communicate with at least one of the electronic device 1404 or the server 1408 via the second network 1499 (e.g., a long-range wireless communication network). According to an embodiment, the electronic device 1401 may communicate with the electronic device 1404 via the server 1408. According to an embodiment, the electronic device 1401 may include a processor 1420, a memory 1430, an input module 1450, a sound output module 1455, a display module 1460, an audio module 1470, a sensor module 1476, an interface 1477, a connection terminal 1478, a haptic module 1479, a camera module 1480, a power management module 1488, a battery 1489, a communication module 1490, a subscriber identification module (SIM) 1496, or an antenna module 1497. In some embodiments, at least one component (e.g., connection terminal 1478) may be omitted from electronic device 1401, or one or more other components may be added in electronic device 1401. In some embodiments, some of the components (e.g., sensor module 1476, camera module 1480, or antenna module 1497) may be implemented as a single component (e.g., display module 1460).

[0142] The processor 1420 may execute, for example, software (e.g., program 1440) to control at least one other component (e.g., hardware or software component) of the electronic device 1401 coupled to the processor 1420, and may perform various data processing or calculations. According to one embodiment, as at least part of the data processing or calculation, the processor 1420 may store a command or data received from another component (e.g., sensor module 1476 or communication module 1490) in the volatile memory 1432, process the command or data stored in the volatile memory 1432, and store the resulting data in the non-volatile memory 1434. According to an embodiment, the processor 1420 may include a main processor 1421 (e.g., a central processing unit (CPU) or an application processor (AP)) or an auxiliary processor 1423 (e.g., a graphics processing unit (GPU), a neural processing unit (NPU), an image signal processor (ISP), a sensor hub processor, or a communication processor (CP)) that may operate independently of the main processor 1421 or in conjunction with the main processor 1421. For example, when the electronic device 1401 includes a main processor 1421 and an auxiliary processor 1423, the auxiliary processor 1423 may be adapted to consume less power than the main processor 1421 or be specific to a specified function. The auxiliary processor 1423 may be implemented separately from the main processor 1421 or as part of the main processor 1421.

[0143] The auxiliary processor 1423 may replace the main processor 1421 when the main processor 1421 is in an inactive (e.g., sleep) state or control at least some of the functions or states related to at least one of the components of the electronic device 1401 (e.g., display module 1460, sensor module 1476, or communication module 1490) together with the main processor 1421 when the main processor 1421 is in an active state (e.g., executing an application). According to an embodiment, the auxiliary processor 1423 (e.g., an image signal processor or a communication processor) may be implemented as a part of another component (e.g., a camera module 1480 or a communication module 1490) that is functionally related to the auxiliary processor 1423. According to an embodiment, the auxiliary processor 1423 (e.g., a neural processing unit) may include a hardware structure specified for artificial intelligence model processing. The artificial intelligence model may be generated by machine learning. Such learning may be performed, for example, by the electronic device 1401 that performs artificial intelligence or via a separate server (e.g., server 1408). The learning algorithm may include, but is not limited to, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. The artificial intelligence model may include multiple artificial neural network layers. The artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q network, or a combination of two or more thereof, but is not limited thereto. The artificial intelligence model may additionally or alternatively include a software structure in addition to a hardware structure.

[0144] The memory 1430 may store various data used by at least one component of the electronic device 1401 (e.g., the processor 1420 or the sensor module 1476). The various data may include, for example, input data or output data of software (e.g., program 1440) and commands related thereto. The memory 1430 may include a volatile memory 1432 or a non-volatile memory 1434.

[0145] The program 1440 may be stored as software in the memory 1430 , and may include, for example, an operating system (OS) 1442 , middleware 1444 , or an application 1446 .

[0146] The input module 1450 may receive a command or data to be used by another component (e.g., the processor 1420) of the electronic device 1401 from outside (e.g., a user) of the electronic device 1401. The input module 1450 may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus).

[0147] The sound output module 1455 can output sound signals to the outside of the electronic device 1401. The sound output module 1455 may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as playing multimedia or playing recordings. The receiver may be used to receive incoming calls. Depending on the embodiment, the receiver may be implemented as being separated from the speaker or as a part of the speaker.

[0148] The display module 1460 can visually provide information to the outside of the electronic device 1401 (e.g., a user). The display module 1460 may include, for example, a display, a hologram device, or a projector, and a control circuit for controlling a corresponding one of the display, the hologram device, and the projector. According to an embodiment, the display module 1460 may include a touch sensor adapted to detect a touch, or a pressure sensor adapted to measure the strength of a force caused by a touch.

[0149] The audio module 1470 can convert sound into an electrical signal, and vice versa. According to an embodiment, the audio module 1470 can obtain sound via the input module 1450, or output sound via the sound output module 1455 or an earphone of an external electronic device (e.g., electronic device 1402) directly (e.g., wired) or wirelessly coupled to the electronic device 1401.

[0150] The sensor module 1476 may detect an operating state (e.g., power or temperature) of the electronic device 1401 or an environmental state (e.g., a user's state) outside the electronic device 1401, and then generate an electrical signal or data value corresponding to the detected state. Depending on the embodiment, the sensor module 1476 may include, for example, a gesture sensor, a gyroscope sensor, an atmospheric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illumination sensor.

[0151] The interface 1477 may support one or more designated protocols to be used for direct (e.g., wired) or wireless coupling of the electronic device 1401 to an external electronic device (e.g., electronic device 1402). According to an embodiment, the interface 1477 may include, for example, a high-definition multimedia interface (HDMI), a universal serial bus (USB) interface, a secure digital (SD) card interface, or an audio interface.

[0152] The connection terminal 1478 may include a connector, and the electronic device 1401 may be physically connected to an external electronic device (e.g., the electronic device 1402) via the connector. According to an embodiment, the connection terminal 1478 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0153] The haptic module 1479 may convert the electrical signal into mechanical stimulation (eg, vibration or movement) or electrical stimulation, which may be recognized by the user via his tactile sense or kinesthetic sense. According to an embodiment, the haptic module 1479 may include, for example, a motor, a piezoelectric element, or an electrical stimulator.

[0154] The camera module 1480 may capture still images or moving images. According to an embodiment, the camera module 1480 may include one or more lenses, an image sensor, an image signal processor, or a flash.

[0155] The power management module 1488 may manage power supplied to the electronic device 1401. According to an embodiment, the power management module 1488 may be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0156] The battery 1489 may supply power to at least one component of the electronic device 1401. According to an embodiment, the battery 1489 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0157] The communication module 1490 can support establishing a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device 1401 and an external electronic device (e.g., electronic device 1402, electronic device 1404, or server 1408), and perform communication via the established communication channel. The communication module 1490 may include one or more communication processors that can operate independently of the processor 1420 (e.g., an application processor (AP)) and support direct (e.g., wired) communication or wireless communication. According to an embodiment, the communication module 1490 may include a wireless communication module 1492 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 1494 (e.g., a local area network (LAN) communication module or a power line communication (PLC) module). A corresponding one of these communication modules can communicate via a first network 1498 (e.g., a short-range communication network 1498, such as Bluetooth TMThe wireless communication module 1492 can communicate with an external electronic device using a wireless communication network 1498 (e.g., a wireless fidelity (Wi-Fi) direct connection or an infrared data association (IrDA)) or a second network 1499 (e.g., a long-distance communication network such as a traditional cellular network, a 5G network, a next-generation communication network, the Internet)) or a computer network (e.g., a LAN or a wide area network (WAN)). These various types of communication modules can be implemented as a single component (e.g., a single chip), or can be implemented as multiple components separated from each other (e.g., multiple chips). The wireless communication module 1492 can use the subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module 1496 to identify and authenticate the electronic device 1401 in the communication network (e.g., the first network 1498 or the second network 1499).

[0158] The wireless communication module 1492 can support 5G networks and next-generation communication technologies (e.g., new radio (NR) access technology) after the 4G network. NR access technology can support enhanced mobile broadband (eMBB), massive machine type communication (mMTC), or ultra-reliable and low-latency communication (URLLC). The wireless communication module 1492 can support high-frequency bands (e.g., mmWave bands) to achieve, for example, high data transmission rates. The wireless communication module 1492 can support various technologies for ensuring performance on high-frequency bands, such as, for example, beamforming, massive multiple input and multiple output (massive MIMO), full-dimensional MIMO (FD-MIMO), array antennas, analog beamforming, or massive antennas. The wireless communication module 1492 can support various requirements specified in the electronic device 1401, an external electronic device (e.g., electronic device 1404), or a network system (e.g., a second network 1499). In accordance with an embodiment, the wireless communication module 1492 may support a peak data rate for implementing eMBB (e.g., 20 Gbps or greater), loss coverage for implementing mMTC (e.g., 1464 dB or less), or U-plane delay for implementing URLLC (e.g., 0.5 ms or less for each of the downlink (DL) and uplink (UL), or 14 ms or less for a round trip).

[0159] The antenna module 1497 may send a signal or power to the outside of the electronic device 1401 (e.g., an external electronic device) or receive a signal or power from the outside of the electronic device 1401 (e.g., an external electronic device). According to an embodiment, the antenna module 1497 may include an antenna including a radiating element including a conductive material or a conductive pattern formed in or on a substrate (e.g., a printed circuit board (PCB)). According to an embodiment, the antenna module 1497 may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication scheme used in a communication network such as the first network 1498 or the second network 1499 may be selected from the plurality of antennas by the communication module 1490 (e.g., a wireless communication module 1492). Then a signal or power may be sent or received between the communication module 1490 and the external electronic device via the selected at least one antenna. According to an embodiment, another component (e.g., a radio frequency integrated circuit (RFIC)) other than the radiating element may be additionally formed as a part of the antenna module 1497.

[0160] According to various embodiments, the antenna module 1497 may form a millimeter wave antenna module. According to an embodiment, the millimeter wave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., millimeter wave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving signals of the specified high frequency band.

[0161] At least some of the above components can be coupled to each other and transmit signals (e.g., commands or data) therebetween via an inter-peripheral communication scheme (e.g., a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)).

[0162] According to an embodiment, a command or data may be sent or received between the electronic device 1401 and the external electronic device 1404 via a server 1408 coupled to the second network 1499. Each of the electronic devices 1402 or 1404 may be a device of the same type or a different type as the electronic device 1401. According to an embodiment, all or some operations to be performed at the electronic device 1401 may be performed at one or more of the external electronic devices 1402, 1404, or 1408. For example, if the electronic device 1401 should automatically perform a function or service, or in response to a request from a user or another device, the electronic device 1401 may request one or more external electronic devices to perform at least a portion of the function or service instead of or in addition to performing the function or service. The one or more external electronic devices receiving the request may perform at least a portion of the requested function or service, or an additional function or additional service related to the request, and transmit the result of the execution to the electronic device 1401. The electronic device 1401 may provide the result as at least a part of the reply to the request with or without further processing the result. To this end, for example, cloud computing, distributed computing, mobile edge computing (MEC) or client-server computing technology can be used. Electronic device 1401 can use, for example, distributed computing or mobile edge computing to provide ultra-low latency services. In an embodiment, external electronic device 1404 may include an Internet of Things (IoT) device. Server 1408 may be an intelligent server using machine learning and / or neural networks. According to an embodiment, external electronic device 1404 or server 1408 may be included in second network 1499. Electronic device 1401 can be applied to intelligent services (e.g., smart homes, smart cities, smart cars, or health care) based on 5G communication technology or IoT-related technologies.

[0163] As described above, an electronic device according to an example embodiment may include: a microphone, a communication circuit, and a processor. According to an example embodiment, the processor may be configured to: recognize a voice command based on recognition of a voice signal obtained through the microphone, the voice command indicating display through a display of a virtual device, the virtual device being registered by using a software application for controlling another device through the electronic device. According to an example embodiment, the processor may be configured to: recognize whether a mixed reality MR device configured to display the virtual device is worn based on the voice command. According to an example embodiment, the processor may be configured to: in response to recognizing that the MR device is worn, based on device information registered as associated with the virtual device by using the software application, obtain an image to be displayed on the display of the virtual device displayed by the MR device. According to an example embodiment, the processor may be configured to: send a signal to the MR device through the communication circuit for displaying the image on the display of the virtual device displayed by the MR device.

[0164] According to example embodiments, the processor may be configured to provide a notification for informing to wear the MR device in response to recognizing that the MR device is not worn.

[0165] According to an example embodiment, the processor may be configured to identify a display location based on the recognition of the voice signal. According to an example embodiment, the processor may be configured to recognize the voice command based on identifying that the display location corresponds to a location of the virtual device registered as associated with the virtual device using the software application.

[0166] According to an example embodiment, the processor may be configured to recognize the voice command based on the recognition of the voice signal, and the voice command further instructs to obtain data of the image from another electronic device registered using the software application. According to an example embodiment, the processor may be configured to request the data from the other electronic device through the communication circuit based on the voice command in response to recognizing that the MR device is worn. According to an example embodiment, the processor may be configured to receive the data from the other electronic device through the communication circuit. According to an example embodiment, the processor may be configured to obtain the image based on the data and the device information.

[0167] According to an example embodiment, the image corresponds to another image displayed by a display of the another electronic device.

[0168] According to an example embodiment, the image may include an image extended from another image displayed through a display of the another electronic device.

[0169] According to an example embodiment, the processor may be configured to, based on recognition of the voice signal, identify that the voice signal indicates display through a display of the other electronic device and display through a display of the virtual device, the other electronic device being registered by using the software application. According to an example embodiment, the processor may be configured to: identify the location of the virtual device based on identifying that the location of the electronic device identified by the communication circuit (240) corresponds to the location of the virtual device registered as associated with the virtual device by using the software application and the location of another electronic device registered as associated with the other electronic device by using the software application.

[0170] The electronic device according to the example embodiment may further include a display. According to the example embodiment, the processor may be configured to recognize the voice command based on the recognition of the voice signal obtained while performing display on the display of the electronic device based on the first data received from the external electronic device through the communication circuit. According to the example embodiment, the processor may be configured to send the device information to the external electronic device through the communication circuit in response to recognizing that the MR device is worn. According to the example embodiment, the processor may be configured to obtain an image by receiving second data obtained based on the device information from the external electronic device via the communication circuit.

[0171] According to an example embodiment, the processor may be configured to recognize that the voice signal indicates display through the display of the virtual device and display of the display of another electronic device registered by using the software application based on recognition of the voice signal. According to an example embodiment, the processor may be configured to recognize that the position of the other electronic device registered as associated with the other electronic device by using the software application and the position of the virtual device registered as associated with the virtual device by using the software application are included in the area. According to an example embodiment, the processor may be configured to: based on recognizing that the position of the other electronic device and the position of the virtual device are included in the area, identify the direction of the display of the other electronic device based on the position of the other electronic device, and identify the direction of the display of the virtual device based on the position of the virtual device. According to an example embodiment, the processor may be configured to recognize the voice command based on the direction of the display of the other electronic device and the direction of the display of the virtual device.

[0172] According to example embodiments, the processor may be configured to receive a signal indicating a state of the MR device from the MR device connected to the electronic device through the communication circuit.According to example embodiments, the processor may be configured to recognize whether the MR device is worn based on the received signal.

[0173] According to example embodiments, the signal indicative of the state of the MR device may include information indicative of a movement of the MR device obtained by a sensor of the MR device.

[0174] According to example embodiments, the signal indicating the state of the MR device may include information indicating a distance between the MR device and an external electronic device obtained by a sensor of the MR device.

[0175] According to an example embodiment, the processor may be configured to, while the connection with the MR device is released, identify another image to be displayed by the display of the virtual device located in the virtual space accessible by using the MR device by using the software application. According to an example embodiment, the processor may be configured to: while the connection with the MR device is released, transmit data for displaying the other image on the display of the virtual device to an external electronic device providing the virtual space through the communication circuit.

[0176] According to an example embodiment, the processor may be configured to recognize another voice command based on recognition of another voice command obtained through the microphone, the another voice command indicating a display in another location different from the location of multiple devices including the virtual device, the multiple devices being registered by the software application. According to an example embodiment, the processor may be configured to: based on the another voice command, display a user interface of the software application for registering another virtual device located at the another location on the display of the electronic device.

[0177] According to an example embodiment, the processor may be configured to obtain device information of another virtual device through a user interface. According to an example embodiment, the processor may be configured to register another virtual device associated with the device information and another location using a software application.

[0178] The electronic device according to various embodiments may be one of various types of electronic devices. The electronic device may include, for example, a portable communication device (e.g., a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, a household appliance, etc. According to an embodiment of the present disclosure, the electronic device is not limited to the above-mentioned electronic devices.

[0179] It should be understood that the various embodiments of the present disclosure and the terms used therein are not intended to limit the technical features set forth herein to specific embodiments, and include various changes, equivalents or replacements of the corresponding embodiments. With regard to the description of the accompanying drawings, similar reference numerals may be used to refer to similar or related elements. It should be understood that, unless otherwise clearly stated in the relevant context, the singular form of the noun corresponding to the project may include one or more things. As used herein, each of the phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "at least one of A, B or C" may include any one or all possible combinations of the items listed together in the corresponding phrase in the phrase. As used herein, terms such as "first" and "second" or "first" and "second" may be used to simply distinguish the corresponding component from another component, and do not limit the component in other aspects (e.g., importance or order). It should be understood that if an element (e.g., a first element) is referred to as being “coupled”, “coupled to”, “connected to”, or “connected to” another element (e.g., a second element) with or without the term “operably” or “communicatively”, the element may be coupled to the other element directly (e.g., by wire), wirelessly, or via a third element.

[0180] As used in conjunction with various embodiments of the present disclosure, the term "module" may include a unit implemented in hardware, software, or firmware, or any combination thereof, and may be used interchangeably with other terms (e.g., "logic," "logic block," "component," or "circuit"). A module may be a single integrated component or its smallest unit or portion suitable for performing one or more functions. For example, according to an embodiment, a module may be implemented in the form of an application specific integrated circuit (ASIC).

[0181] Various embodiments as described herein may be implemented as software (e.g., program 1440) including one or more instructions stored in a storage medium (e.g., internal memory 1436 or external memory 1438), which may be read by a machine (e.g., electronic device 1401). For example, a processor (e.g., processor 1420) of a machine (e.g., electronic device 1401) may call at least one of the one or more instructions stored in the storage medium and execute it with or without one or more other components under the control of the processor. This allows the machine to be operated to perform at least one function according to at least one instruction called. One or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Among them, a "non-transitory" storage medium is a tangible device and may not include a signal (e.g., an electromagnetic wave), but the term does not distinguish between a location where data is semi-permanently stored in a storage medium and a location where data is temporarily stored in a storage medium.

[0182] According to an embodiment, the method according to various embodiments of the present disclosure may be included and provided in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read-only memory (CD-ROM)), or via an application store (e.g., a Play Store). TM ) online distribution (e.g., download or upload), or directly between two user devices (e.g., smartphones). If distributed online, at least a portion of the computer program product may be temporarily generated or at least temporarily stored in a machine-readable storage medium, such as a memory of a manufacturer's server, an application store's server, or a relay server.

[0183] According to various embodiments, each of the above-mentioned components (e.g., a module or a program) may include a single entity or multiple entities, and some of the multiple entities may be separately arranged in different components. According to various embodiments, one or more of the above-mentioned components may be omitted, or one or more other components may be added. Alternatively or additionally, multiple components (e.g., modules or programs) may be integrated into a single component. In this case, according to various embodiments, the integrated component may still perform one or more functions of each of the multiple components in the same or similar manner as performed by a corresponding one of the multiple components before integration. According to various embodiments, the operations performed by a module, a program or another component may be performed sequentially, in parallel, repeatedly or heuristically, or one or more of the operations may be performed or omitted in a different order, or one or more other operations may be added.

Claims

1. An electronic device (101), include: Microphone (230); Communication circuit (240); as well as The processor (210) is configured to: Recognizing a voice command based on recognition of a voice signal obtained through the microphone (230), the voice command instructing display through a display of a virtual device (105) registered by using a software application for controlling another device through the electronic device (101); Based on the voice command, identifying whether a mixed reality MR device (102) for displaying the virtual device (105) is worn; In response to recognizing that the MR device (102) is worn, obtaining an image to be displayed on the display of the virtual device (105) displayed by the MR device (102) based on device information registered as associated with the virtual device (105) by using the software application; as well as A signal for displaying the image on the display of the virtual device (105) displayed by the MR device (102) is sent to the MR device (102) through the communication circuit (240).

2. The electronic device according to claim 1, in, The processor (210) is configured to provide a notification for informing to wear the MR device (102) in response to recognizing that the MR device (102) is not worn.

3. The electronic device according to claim 1, in, The processor (210) is configured to: identifying a display position based on the recognition of the voice signal; and The voice command is recognized based on recognizing that the display location corresponds to a location of the virtual device (105) that is registered as being associated with the virtual device (105) using the software application.

4. The electronic device according to claim 1, in, The processor (210) is configured to: Based on the recognition of the voice signal, identifying the voice command, the voice command further instructing to obtain data of the image from another electronic device (103) registered by using the software application; In response to recognizing that the MR device (102) is worn, requesting the data from the other electronic device (103) through the communication circuit (240) based on the voice command; receiving the data from the other electronic device (103) via the communication circuit (240); and The image is obtained based on the data and the device information.

5. The electronic device according to claim 4, in, The image corresponds to another image displayed by a display of the other electronic device (103).

6. The electronic device according to claim 4, in, The image is an image extended from another image displayed by a display of the another electronic device (103).

7. The electronic device according to claim 1, in, The processor (210) is configured to: Based on the recognition of the voice signal, recognizing that the voice signal indicates display through a display of the other electronic device (103) and display through a display of the virtual device (105), the other electronic device (103) is registered by using the software application; as well as Based on identifying that the location of the electronic device (101) identified by the communication circuit (240) corresponds to the location of the virtual device (105) registered as associated with the virtual device (105) by using the software application and the location of another electronic device (103) registered as associated with another electronic device (103) by using the software application, the location of the virtual device (105) is identified.

8. The electronic device according to claim 1, further comprising: include: a display (220), Wherein, the processor (210) is configured to: recognizing the voice command based on recognition of the voice signal obtained while performing display on the display (220) of the electronic device (101) based on first data received from an external electronic device through the communication circuit (240); In response to recognizing that the MR device (102) is worn, transmitting the device information to the external electronic device through the communication circuit (240); and An image is obtained by receiving second data obtained based on the device information from an external electronic device via a communication circuit (240).

9. The electronic device according to claim 1, in, The processor (210) is configured to: Based on the recognition of the voice signal, identifying the voice signal indicates display through the display of the virtual device (105) and display of the display of another electronic device (103) registered by using the software application; identifying that a location of the other electronic device (103) registered as associated with the other electronic device (103) by using the software application and a location of the virtual device (105) registered as associated with the virtual device (105) by using the software application are included in a region; Based on identifying that the position of the other electronic device (103) and the position of the virtual device (105) are included in the area, identifying the direction of the display of the other electronic device (103) based on the position of the other electronic device (103), and identifying the direction of the display of the virtual device (105) based on the position of the virtual device (105); and The voice command is recognized based on the orientation of the display of the other electronic device (103) and the orientation of the display of the virtual device (105).

10. The electronic device according to claim 1, in, The processor (210) is configured to: receiving a signal indicating a status of the MR device (102) from the MR device (102) connected to the electronic device (101) through the communication circuit (240); and Whether the MR device (102) is worn is identified based on the received signal.

11. The electronic device according to claim 10, in, The signal indicative of the state of the MR device (102) includes information indicative of movement of the MR device (102) obtained by sensors of the MR device (102).

12. The electronic device according to claim 10, in, The signal indicative of the state of the MR device (102) includes information obtained by a sensor of the MR device (102) indicating a distance between the MR device (102) and an external electronic device.

13. The electronic device according to claim 1, in, The processor (210) is further configured to: identifying, by using the software application, another image to be displayed through the display of the virtual device (105) located in a virtual space accessible by using the MR device (102) while the connection with the MR device (102) is released; and While the connection with the MR device (102) is released, data for displaying the other image on the display of the virtual device (105) is transmitted to an external electronic device providing the virtual space through the communication circuit (240).

14. The electronic device according to claim 1, further comprising: include: a display (220), Wherein, the processor (210) is further configured to: identifying another voice command based on recognition of another voice command obtained through the microphone, the other voice command instructing a display in another location different from the location of a plurality of devices including the virtual device (105), the plurality of devices being registered with the software application; and Based on the other voice command, the display (220) of the electronic device (101) A user interface of the software application for registering another virtual device (105) located at the other location is displayed.

15. A method performed in an electronic device (101) comprising a microphone (230) and a communication circuit (240), the method include: Recognizing a voice command based on recognition of a voice signal obtained through the microphone (230), the voice command instructing display through a display of a virtual device (105) registered by using a software application for controlling another device through the electronic device (101); Based on the voice command, identifying whether a mixed reality MR device (102) for displaying the virtual device (105) is worn; In response to recognizing that the MR device (102) is worn, obtaining an image to be displayed on the display of the virtual device (105) displayed by the MR device (102) based on device information registered as associated with the virtual device (105) by using the software application; as well as A signal for displaying the image on the display of the virtual device (105) displayed by the MR device (102) is sent to the MR device (102) through the communication circuit (240).

Images