System and method for casting content

Abstract

To provide a method, a system and a program that enable a content to be transferred from a user device to an external device for output of the content by the external device.SOLUTION: In a system, an external device (television set 420) may be detected in a physical space 400, and is identified based upon a previous connection with a user device (handheld device 120), based upon a shared network or shared system of connected devices including the user device, and based upon image information captured by the user device and previously stored anchoring information identifying the external device, and then selected for potential output of a content based upon previously stored configuration information related to the external device. The identified external device outputs the transferred content in response to a user verification input verifying that the content is to be output.SELECTED DRAWING: Figure 4A

Description

Background Art

[0001] Background Connected, networked, or compatible devices allow users to consume, share, and control content using different devices. Some systems rely on a series of user inputs to scan for and detect compatible devices, initiate content sharing and / or mirroring, and control compatible devices. As a result, the user experience can be complex and cumbersome, and users may be hesitant to utilize this functionality.

Summary of the Invention

[0002] Summary In one general aspect, a method implemented by a computer includes a processor of a user device executing content on the user device, the user device detecting at least one external device within a set of settings of the user device, identifying at least one external device for output of the content executed by the user device based on output capabilities of the at least one external device and output characteristics of the content executed by the user device, and transferring execution of the content from the user device to the at least one external device for output by the at least one external device.

[0003] In some implementations, detecting at least one external device within the set range of the user device includes detecting at least one external device within the field of view of the user device. In some implementations, identifying at least one external device includes detecting physical features in the image information captured by the user device, identifying the position of the user device based on the detected physical features, and identifying at least one external device based on the identified position of the user device. In some implementations, the user device includes a handheld device and a head-mounted display device, executing content includes executing content on the handheld device, and detecting physical features in the image information includes detecting physical features in the image information captured within the field of view of the head-mounted display device. In some implementations, identifying the position of the user device based on the detected physical features includes identifying the position of the user device based on a comparison of features pre-stored in a database and accessible to the user device via a server.

[0004] In some implementations, detecting at least one external device within the set range of the user device includes at least one of detecting at least one external device as a previously connected device, detecting at least one external device based on an electromagnetic wave signal, detecting at least one external device within a shared network, or detecting at least one external device within a network of previously connected devices. In some implementations, detecting at least one external device within the set range of the user device includes detecting at least one external device within the field of view of the user of the user device.

[0005] In some implementation examples, for output by at least one external device, transferring the execution of content from a user device to at least one external device involves, before transferring the content from the user device to at least one external device, outputting a user verification request and receiving a user verification input for verifying the transfer of the content from the user device to at least one external device, and in response to receiving the user verification input, transferring the execution of the content from the user device to at least one external device. In some implementation examples, outputting a user verification request includes outputting at least one of a visual request or an acoustic request, and receiving a user verification input includes receiving at least one of a touch input or an acoustic input. This includes outputting a user verification request, receiving a user verification input for verifying the transfer of content from the user device to at least one external device, and in response to receiving the user verification input, transferring the execution of the content from the user device to at least one external device. In some implementation examples, outputting a user verification request includes outputting at least one of a visual request or an acoustic request, and receiving a user verification input includes receiving at least one of a touch input or an acoustic input.

[0006] In other general aspects, the system includes an output system including an audio output device and an image output device, a sensing system, at least one processor, and a memory. The memory stores instructions that, when executed by at least one processor, cause the at least one processor to output content to be executed by the at least one processor by the output system, detect at least one external device within the detection range of the sensing system by the sensing system, identify at least one external device for output of the content to be executed by the user device based on the output capabilities of the at least one external device and the output characteristics of the content output by the output system, and transfer the execution of the content to the at least one external device for output by the at least one external device.

[0007] In some implementation examples, when detecting at least one external device, the instruction causes at least one processor to detect at least one external device based on the image information captured by the sensing system, detect at least one external device as a previously connected device, detect at least one external device based on an electromagnetic wave signal, detect at least one external device within a shared network, or detect at least one external device within a network of previously connected devices, and execute at least one of them. In some implementation examples, when detecting at least one external device based on the image information captured by the sensing system, the instruction causes at least one processor to detect physical features in the image information, identify a location based on a comparison between the detected physical features and the pre-stored features in a database accessible to the system via a server, and identify at least one external device based on the identified location.

[0008] In some implementation examples, when transferring the execution of content to at least one external device, the instruction causes at least one processor to output a user verification request before transferring the content to at least one external device, receive a user verification input for verifying the transfer of the content to at least one external device, and in response to receiving the user verification input, transfer the execution of the content to at least one external device. In some implementation examples, the user verification request includes at least one of a visual request or an acoustic request, and the user verification input includes at least one of a touch input or an acoustic input. In some implementation examples, the output system, the sensing system, at least one processor, and the memory are installed in at least one of a handheld electronic device or a head-mounted display device operable within a network.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2A

Figure 2B

Figure 2C

Figure 2D

Figure 2E

Figure 3

Figure 4A

Figure 4B

Figure 4C

Figure 4D

Figure 4E

Figure 4F

Figure 5

Figure 6

Best Mode for Carrying Out the Invention

[0010] Detailed Description A user can consume content using a number of different types of devices, such as, for example, wearable devices like handheld devices (such as smartphones), hand and / or wrist-worn devices (such as smartwatches, smart bracelets, smart rings, etc.), head-worn devices (such as smart glasses, goggles, headsets, etc.), ear-worn devices, neck-worn strap devices, other mobile devices (such as tablet computing devices, laptop computing devices, etc.), desktop computing devices, smart TVs, smart speakers, etc. The systems and methods according to the implementation examples described herein enable sharing and transfer of the execution of content executed and output by a user device to an external device detected from the user device for execution and output by the external device in response to the detection of the external device. For example, the execution of the content can be transferred or shared for output by the external device in response to detection of an external device within a set proximity or within a set range of the user device, detection of an external device within the field of view of the user device, detection of an external device based on image data collected by the user device, detection of an external device within a shared local network, detection of a set signal strength associated with the external device, and other such detection conditions. In some implementation examples, the detection of the external device for sharing or transfer of the output of the content is performed without user intervention. In some implementation examples, the content is shared or transferred for output by the detected external device without user intervention. In some implementation examples, the content is shared or transferred for output by the detected external device in response to user input indicating verification or confirmation. In some implementation examples, the transfer of the content from the user device to the external device for detection and / or execution by the external device may be performed with limited user intervention to enhance the user experience when consuming the content within the system of the connected devices.

[0011] Depending on the situation, the user experience can be improved by sharing the execution of the content or transferring it to the detected external device for output by the detected external device. For example, a user viewing content, such as video content, on a mobile device such as a smartphone may prefer to view the content on an alternative output device with greater image output capabilities, such as a TV display screen or a computer display screen, if a larger image output device is available. Similarly, a user listening to music through, for example, earphones, a head-mounted wearable audio output device, etc. may prefer to listen to the music through an alternative output device, such as a smart speaker, a sound system, etc., if an alternative audio output device is available. If the identification and transfer of the content for output by the alternative output device is facilitated by, for example, image recognition based on the detected previous connections of the device, the shared network, or the detection of the device within the system of previously connected devices, and the stored anchor information associated with the alternative output device, and the amount of user interaction and input required to complete the transfer for the execution of the content by the alternative output device can be reduced, the user experience can be further improved. Therefore, the term output capability can be used to refer to the quality with which an external device can execute a task or a set of tasks, and / or to distinguish and / or uniquely define the output of images (still and / or moving) and / or audio output of the external device. The term content output characteristics can be used to refer to the specific type of output data (e.g., image data and / or audio data) and / or the amount of data associated with the output.

[0012] FIG. 1 is a diagram showing a user in relation to an exemplary system 100 according to an implementation example described in this specification. In the example shown in FIG. 1, the user wears, for illustrative and diagrammatic purposes only, a wearable device 110 in the form of an exemplary head mounted display (HMD) device 110 or smart glasses 110, a wearable device 130 in the form of an exemplary wrist-worn device 130 or smart watch 130, and a wearable device 140 in the form of an exemplary ear-worn device 140 or earphones 140. In the example shown in FIG. 1, the user holds, for illustrative and diagrammatic purposes only, a handheld device 120 such as a smartphone 120.

[0013] The exemplary system 100 may include one or more computing devices and / or electronic devices that can communicate with each other, for example, via network 195 and / or via alternative network(s) and / or directly. Exemplary client devices or user devices may include, for example, exemplary devices 110, 120, 130, 140 worn and / or operated by a user, and other electronic devices including, for example, a display screen 150 / TV monitor 150, an audio output device 155 / smart speaker 155, a laptop or notebook computing device 160, a tablet computing device 170, a desktop computing device 180, and other such devices. The devices can access one or more external resources 190 via network 195. The external resources 190 may include, for example, a server that provides access to one or more databases and the like.

[0014] Figure 2A is a front view of an exemplary wearable device 110 or HMD 110 that a user wears in FIG. 1. FIG. 2B is a front view of an exemplary handheld device 120 or smartphone 120 that a user holds in FIG. 1. FIG. 2C is a front view of an exemplary wearable device 130 or smartwatch 130 that a user wears in FIG. 1. FIG. 2D is a perspective view of an exemplary wearable device 140 or earphone 140 that a user wears in FIG. 1. FIG. 2E is a perspective view of another exemplary head-mounted wearable device 110'.

[0015] The wearable device 110 in the form of the HMD 110 or smart glasses 110 of the example shown in FIG. 2A may include a frame 111, and a display device 112 is coupled to the frame 111. In some embodiments, an audio output device 113 is coupled to the frame 111. In some embodiments, a touch surface 114 enables user control and input of the HMD 110. The HMD 110 may include a sensing system 116 including various sensing system devices and a control system 117 including various control system devices to facilitate the operation of the HMD 110. The control system 117 may include a processor 119 operably coupled to components of the control system 117 and a communication module 115 that enables communication with external devices and / or networks. Further, the HMD 110 may include an image sensor 118 (i.e., camera 118), a depth sensor, a light sensor, and other such sensing devices. In some embodiments, the image sensor 118 or camera 118 can capture still images and / or videos, patterns, features, and light, etc.

[0016] ​Similarly, the wearable device 110’ shown in FIG. 2E is shown to be attached to glasses worn by a user for use as a head-mounted wearable device 110’. However, the wearable device 110’ can be attached to, for example, earphones, straps, bracelets, and other such devices that can be worn by a user. In some implementations, the wearable device 110’ includes an audio input and / or output device 113’ (i.e., a microphone and / or a speaker), an image sensor 118’ (i.e., a camera 118’), a depth sensor, a light sensor, and other such sensing devices. In some implementations, a touch input surface 114’ enables user control and input of the wearable device 110’. The wearable device 110’ may include a control system 117’ and a processor 119’ to facilitate the operation of the wearable device 110’, and a communication module 115’ to enable communication with external devices and / or a network.

[0017] In the example shown in FIG. 2B, the wearable device 130 in the form of a smartwatch 130 may include an interface device 131. In some embodiments, the interface device 131 functions as an output device that includes, for example, a display area 132 capable of outputting information to the user. In some embodiments, the interface device 131 functions as an input device that includes, for example, a touch surface 133 through which the interface device 131 can receive touch input from the user. In some embodiments, the interface device 131 can function as both an input device and an output device. The wearable device 130 may include a sensing system 136 that includes various sensing system devices. The wearable device 130 may include a control system 137 that includes various control system devices, a communication module 135 that enables communication with external devices and / or networks, and a processor 139 to facilitate the operation of the device 130. The wearable device 130 may also include an image sensor 138 (i.e., a camera 138), a depth sensor, a light sensor, and other such sensing devices. In some embodiments, the image sensor 138 or camera 138 can capture still images and / or videos, patterns, features, light, etc.

[0018] In the example shown in FIG. 2B, the handheld device 120 in the form of a smartphone 120 may include an interface device 121. In some embodiments, the interface device 121 functions as an output device that includes, for example, a display area 122 capable of outputting information to the user. In some embodiments, the interface device 121 functions as an input device that includes, for example, a touch surface 123 through which the interface device 121 can receive touch input from the user. In some embodiments, the interface device 121 functions as both an input device and an output device. The handheld device 120 includes a sensing system 126 that includes various sensing system devices It is possible. The handheld device 120 may include a control system 127 that includes various control system devices to facilitate the operation of the handheld device 120, a communication module 125 that enables communication with external devices and / or a network, and a processor 129. The handheld device 120 may also include an image sensor 128 (i.e., a camera 128), a depth sensor, a light sensor, and other such sensing devices. In some implementations, the image sensor 128 or the camera 128 can capture still images and / or videos, patterns, features, light, etc.

[0019] The wearable device 140 in the form of the earphone 140 is shown in FIG. 2D. The wearable device 140 may include an audio device 143. In some implementations, the audio device 143 functions as an audio output device (i.e., a speaker) and an audio input device (i.e., a microphone). In some implementations, the touch input surface 144 enables user control and input of the wearable device 140, etc. The wearable device 140 may include a control system 147 and a processor 149 to facilitate the operation of the wearable device 140, and a communication module 145 that enables communication with external devices and / or a network.

[0020] FIG. 3 is a block diagram of an exemplary system 300 that can implement the concepts described herein. The block diagram shown in FIG. 3 shows, for example, that a first electronic device 310, such as one of the exemplary user devices 110, 120, 130, 140 shown in FIG. 1, is communicating with a second electronic device 315, such as one of the exemplary external devices 150, 160, 170, 180, 190 shown in FIG. 1.

[0021] The first electronic device 310 may include a sensing system 360 and a control system 370. The sensing system 360 may include one or more different types of sensors, such as, for example, optical sensors, audio sensors, image sensors, distance / proximity sensors, and / or other sensors and / or different combinations of sensors. In some implementations, the sensing system 360 includes an image sensor and / or an optical sensor. The control system 370 may include, for example, power / suspend control device(s), audio and video control device(s), optical control device(s), and / or other such devices and / or different combinations of devices. The sensing system 360 and / or the control system 370 may include more or fewer devices depending on the particular implementation. The electronic device 310 may include a processor 390 that communicates with the sensing system 360 and the control system 370. The processor 390 may process inputs received from the sensing system 360, such as, for example, images and / or optical scans captured by the image sensor / optical sensor. The first electronic device 310 may include an input system 340 that can receive user input processed by the processor 390 and output by the output system 350 under the control of the control system 370. The input system 340 may include various types of input devices, such as, for example, a touch input surface, an audio input device capable of receiving audio input (e.g., an audio sensor included in the sensing system 360, or a microphone), a gesture recognition device (e.g., including images captured by an image sensor(s) of the sensing system 360 and processed by the processor 390), and other such input devices. The output system 350 may include various types of output devices, such as, for example, display device(s), audio output device(s), or speakers, physical and / or tactile output devices, and other such output devices.The first electronic device 310 may include a memory 380 and a communication module 330 that enables communication between the first electronic device 310 and one or more other external devices (such as, for example, a second electronic device 315), a network, a server, etc.

[0022] The second electronic device 315 may include a sensing system 365 that includes one or more different types of sensors and a control system 375 that includes a control module for controlling the operation of the second electronic device 315. The number and / or combination of components of the sensing system 365 and / or the control system 375 may vary depending on the particular embodiment. The second electronic device 315 may include a processor 395 that communicates with the sensing system 365 and the control system 375, whereby the processor 395 can process the input received from the sensing system 360. The second electronic device 315 may include an input system 345 that receives an input that is processed by the processor 395 and output by an output system 355 under the control of the control system 375. The output system 355 may include various types of output devices such as, for example, a display device(s), an audio output device(s), or a speaker, a physical and / or tactile output device, and other such output devices. In some implementations, the second electronic device 315 includes a memory 385 and a communication module 335, and the communication module 335 enables communication between the second electronic device 315 and one or more other external devices (such as, for example, the first electronic device 310), a network, a server, etc.

[0023] Figures 4A through 4F are diagrams showing the detection and / or identification of an external device, and the transfer of execution of content or the casting of content to the external device identified from a user device, according to an implementation example described in this specification. In the example described with respect to Figures 4A through 4F, the user device refers to one or more devices used by a user to access content, transfer the execution of content, or cast content to an external device, and the external device refers to one or more devices within a physical space configured to receive and output content from the user device(s). In the example described with respect to Figures 4A through 4F, for purposes of illustration only, the handheld device 120 and the wearable device 110 (in the form of smart glasses 110) are used as exemplary user devices, and the television 420 is used as an exemplary external device that receives content from the user device. The principles described apply to other devices and / or combinations of devices that function as user devices, as well as other devices and / or combinations of devices that function as external devices.

[0024] FIG. 4A is a third-person view of a user approaching an exemplary physical space 400. In the example shown in FIG. 4A, the user is wearing an exemplary wearable device 110 (i.e., HMD 110 / smart glasses 110) and an exemplary wearable device 130 (i.e., smart watch 130). In the example shown in FIG. 4A, the user is accessing content 410 using a handheld device 120. In the exemplary scenario shown in FIG. 4A, the content 410 would be output to the display area 122 of the handheld device 120, but in FIG. 4A, the content 410 is shown in an exterior pane of the handheld device 120 for ease of explanation and illustration only. In this example, the content 410 is image or video content. The principles described herein can be applied to other types of content and / or to other kinds of user devices that provide users with access to content.

[0025] As shown in FIG. 4B, as the user moves further into the physical space 400, external devices within the physical space 400 may be detected. For example, external devices capable of communicating with the user device(s) (e.g., handheld device 120 and / or wearable device(s) 110, 130) being used by the user may be detected. The detection and / or identification of the external device(s) may enable an exchange of information between the user device(s) and the detected external device(s) to provide the user with information for output by the external device(s). The present invention may enable a user to transfer or share or cast content to a detected external device(s) to provide an improved user experience.

[0026] In some implementations, the external device(s) is / are detected based on a previously established connection or pairing between the user device and the detected external device(s). In some implementations, the external device(s) is / are detected based on a detected signal, such as a short-range signal, e.g., a Wi-Fi (registered trademark) signal or a Bluetooth (registered trademark) signal, an electromagnetic signal, etc., emitted by the external device(s). In some implementations, the external device(s) is / are detected based on the detected signal strength. In some implementations, the external device(s) is / are detected in response to a determination that the user device(s) and the external device(s) are within the same local network. In some implementations, the user device(s) (e.g., the handheld device 120 and / or the wearable device 110) and the external device(s) include an ultra-wideband (UWB) chip that enables detection of communication between the user device(s) and the external device(s) for information exchange. In some implementations, the external devices, features, and / or elements in the physical space can be detected in the image information captured by the user device(s). The image information captured by the user device(s) can be compared with pre-stored anchoring information to identify the features that define the physical space 400. This can be used to identify the physical space 400, identify known external device(s) within the identified physical space 400, and / or detect, locate, and identify external device(s) within the physical space 400.

[0027] Returning to FIG. 4B, the exemplary physical space 400 includes a TV 420 attached to two physical boundaries of the physical space 400 that define a corner 490, i.e., the intersection of the walls. The display of the computing device 440 is placed on the desk 450 on the first side of the chair 460. A lamp 480 is placed on the first side of the sofa 470. An audio output device 430 or a smart speaker 430 is placed on the table between the respective second sides of the chair 460 and the sofa 470. The features and devices in the exemplary physical space 400 shown in FIGS. 4A and 4B are provided for illustrative and diagrammatic purposes only. The principles described herein can be implemented in situations where more or fewer features, elements, devices, etc. are available and / or are arranged differently.

[0028] As described above, when the user moves within the physical space 400, various external devices such as, for example, the television 420 and the smart speaker 430 can be detected by the user device (in this example, the handheld device 120 and / or the HMD 110). For example, the external devices (in the form of the television 420 and the smart speaker 430 in this example) can be detected based on previous connections or pairings with the user device(s) (in the form of the handheld device 120 and the HMD 110 in this example), detection of the user device(s) and the external device(s) in the same local network (e.g., a home network or a local area network), signals emitted by the external device(s) and detected by the user device(s), etc. In some implementations, the detected external device(s) are identified by related functions. For example, the television 420 can be identified as being capable of outputting image content (still images and / or video content along with audio content). The smart speaker 430 can be identified as being capable of outputting audio content. A connection is established between one (or more) of the user device(s) and the identified external device(s), and content is transferred from the user device(s) to the identified external device(s) for execution by the identified external device(s). can be facilitated.

[0029] Depending on the situation, when the user enters the physical space 400, various features, elements, devices, etc. are captured within the field of view V of the user device in the form of the HMD 110 in the example shown in FIG. 4C. In some implementations, the image information captured by the user device is used to detect and identify external devices within the physical space 400. For example, the image information captured by the user device can be used to identify external devices within the physical space that are compatible with content transfer or sharing or casting from the user device and / or capable of outputting the content transferred from the user device to the external device.

[0030] The exemplary physical space 400 includes a number of examples of substantially consistent features, which are detectable in images captured by the image sensor of the user device and can be used to identify the physical space 400 and locate external devices within the physical space 400. For example, the TV 420 can be identified based on its known placement in the user's living room (in this example, the physical space 400), which was established, for example, during the setup and configuration process of the user's home devices. In some implementations, the TV 420 can be identified based on the detection of the corner 490 of the physical space 400 and its positioning above the chair 460 and the smart speaker 430. These physical features related to the area surrounding the TV 420 can be substantially consistent physical features that define the anchoring information related to the TV 420 in the physical space, thus facilitating the subsequent identification of the TV 420 in the physical space 400. FIG. 4D shows a view of the TV 420 through the HMD 110, including these substantially consistent features. Similarly, the smart speaker 430 can be identified based on its known placement proximate to the corner 490 between the sofa 470 and the chair 460. This type of anchoring information may be collected and stored, for example, in a database accessible through a server via a network as described above. The user can access the stored anchoring information for the identification of the physical space 400 and the location of external devices within the physical space 400. The user can transfer, or share, or cast content to the thus detected and located external devices for execution by the external devices.

[0031] In the example shown in FIGS. 4A-4F, a user is consuming content such as video content output by a handheld device 120. When the user enters the physical space 400, one or more external devices within the physical space 400 that are adapted to receive the content 410 from the handheld device 120 and output the content 410 can be detected and identified in the manner described above with respect to FIGS. 4B and / or 4C. That is, one or more external devices (s) equipped to output the content 410 can be detected. In response to the detection and identification of the external device in the physical space 400, the system may determine that a television 420 (in this example, the external device) can enable an improved viewing of the content 410 currently being output by the handheld device 120 (in this example, the user device). The system can make this determination based on, for example, anchor information including previous connections and pairings, previous configurations of devices within the network, pre-stored input / output capabilities or characteristics of the television 420 (compared to the output characteristics of the content 410), pre-stored processing capabilities or characteristics of the television 420.

[0032] In this example, the content 410 output by the handheld device 120 can be video content 410 that includes image content and audio content. Accordingly, based on the characteristics associated with the content 410, the television 420 can transfer or share or cast the video content 410 from the handheld device 120 for output. It can be identified as an external device within the physical space 400 that is likely to be affected. In some embodiments, the system may generate an inquiry 425, or a prompt 425, or a verification request 425, as shown in FIG. 4E. The verification request 425 may request the user to confirm that the content 410 should be transferred or shared or cast to the TV 420 before casting, in order to avoid inadvertent or careless casting of the content to the external device. In the example shown in FIG. 4E, for purposes of illustration, the verification request 425 is a visual output by the TV 420. In some embodiments, the verification request 425 is output in different forms. For example, the verification request 425 can be an audio output, an audio output combined with a video output, and a physical output, etc. In response to the verification request 425, the user may enter a verification input. The verification input may include, for example, an input on the HMD 110, an input on the handheld device 120, etc. For example, the verification input may include a touch or tap input on the touch surface 114 of the HMD 110, a touch surface of the handheld device 120, or other such user inputs. The verification input may include an audible input or a voice input detected by an audio input device or a microphone of the external device. As shown in FIG. 4F, in response to the user verification input, the content 410 output by the handheld device 120 can be output by the TV 420.

[0033] In some embodiments, the content from the user device (in this example, the handheld device 120) may be transferred or cast to the external device (in this example, the TV 420) for execution without user intervention. That is, in some embodiments, without the verification request 425 and the user verification input described above with respect to FIG. 4E, the content from the user device (in this example, the handheld device 120) may be transferred or shared or cast to the external device (in this example, the TV 420) for execution by the external device.

[0034] FIG. 5 is a flowchart showing an exemplary method 500 for detecting external device(s) and transferring content execution and output to the detected external device(s), according to an implementation example described herein. Content, such as content 410 output by a user device in the form of the handheld device 120 described above with respect to FIGS. 4A - 4F, can be executed and output by a first device (block 510). One or more external device(s) can be detected (blocks 520, 530). The detected external device(s) can be communicable with the first device via, for example, a network or other types of connections. In some implementation examples, the external device(s) are detected based on, for example, a previous connection or pairing to the first device, a shared network with the first device, shared signals and / or signal strength, image information extracted from an image captured by the first device and / or another user device and matching pre-stored anchor information. The system can determine that one or more of the detected external device(s) are configured for output of the content currently output by the first device (block 540) and can select one of the detected external devices for output of the content.

[0035] For example, as described above with respect to FIGS. 4A - 4F, the system can determine which of the detected external devices detected in the physical space 400 are capable of outputting the content 410 currently output by the handheld device 120. In the example above, the content 410 output by the handheld device 120 includes at least image content and may also include audio content. Thus, the system can determine that the television 420 is most appropriately configured (among the external devices detected in the physical space 400) for output of the video content 410. For example, the audio output device 430 or smart speaker 430 may have limited or no image output capabilities. Similarly, the computing device 440 may have audio output capabilities The force may be limited or not exist at all. Thus, the system may select the TV 420 as the second device for output of the content 410.

[0036] In some embodiments, the system outputs a verification request (block 550) before transferring the execution of the content from the first device to the second device, for example, to avoid unintentional transfer or sharing or casting of the content to an external device. In response to a user input that verifies the transfer of the content from the first device to the second device (block 560), the content may be transferred to the second device for output by the second device (block 570).

[0037] The systems and methods according to the embodiments described herein enable sharing and transferring the execution of content output by a user device from the user device to an external device for execution and output by the external device. The execution of the content may be transferred or shared for output by the external device in response to detection of an external device within a set proximity or range of the user device, detection of an external device within the field of view of the user device, detection of an external device based on image data collected by the user device, detection of an external device within a shared local network, detection of a set signal strength associated with the external device, electromagnetic signals, and other such detection conditions. The detection of the external device and / or the transfer of the content from the user device to the external device for execution by the external device may be performed with limited user intervention to improve the user experience when consuming the content within the system of connected devices.

[0038] FIG. 6 shows an example of a general-purpose computer device 600 and general-purpose mobile computer devices 650, 690 that can be used with the techniques described herein. Computing device 600 is intended to represent various forms of digital computers, such as a laptop, desktop, tablet, workstation, personal digital assistant, television, server, blade server, mainframe, and other appropriate computing devices. For example, computing device 600 may be, and / or may be used as, the server mentioned above. Computing device 650 is intended to represent various forms of mobile devices, such as a personal digital assistant, cellular phone, smartphone, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are intended to be exemplary only, and are not intended to limit embodiments of the invention described and / or claimed herein.

[0039] The computing device 600 includes a processor 602, a memory 604, a storage device 606, a high-speed interface 608 that connects to the memory 604 and the high-speed expansion port 610, and a low-speed interface 612 that connects to the low-speed bus 614 and the storage device 606. The processor 602 can be a semiconductor-based processor. The memory 604 can be a semiconductor-based memory. Each of the components 602, 604, 606, 608, 610, and 612 can be interconnected using various buses and can be mounted on a common motherboard or in other suitable manners as appropriate. The processor 602 is capable of processing instructions for execution within the computing device 600, including instructions stored in the memory 604 or the storage device 606, to display graphic information for a GUI on an external input / output device such as a display 616 coupled to the high-speed interface 608. In other embodiments, multiple processors and / or multiple buses may be used as appropriate, along with multiple memories and multiple types of memories. Also, multiple computing devices 600 may be connected, with each device providing a portion of the required operations (e.g., as a server bank, a group of blade servers, or a multi-processor system).

[0040] The memory 604 stores information within the computing device 600. In one embodiment, the memory 604 is one or more volatile memory units. In another embodiment, the memory 604 is one or more non-volatile memory units. Also, the memory 604 may be another form of computer-readable medium, such as a magnetic disk or an optical disk.

[0041] Storage device 606 can provide mass storage for computing device 600. In one implementation, storage device 606 can be a floppy (registered trademark) disk device, a hard disk device, an optical disk device, or a tape device, etc., or an array of devices including a flash memory or other similar solid state memory devices, or a device in a storage area network or other configuration. The computer program product is tangibly embodied in an information carrier. The computer program product may also include instructions that, when executed, perform one or more of the methods as described above. The information carrier is a computer-readable medium or a machine-readable medium such as memory 604, storage device 606, or memory on processor 602.

[0042] High-speed controller 608 manages the bandwidth-intensive operations of computing device 600, and low-speed controller 612 manages the lower bandwidth-intensive operations. Such an assignment of functions is merely exemplary. In one implementation, high-speed controller 608 is coupled to high-speed expansion port 610 that can receive memory 604, display 616 (e.g., via a graphics processor or accelerator), and various expansion cards (not shown). In this implementation, low-speed controller 612 is coupled to storage device 606 and low-speed expansion port 614. The low-speed expansion port that may include various communication ports (e.g., USB, Bluetooth (registered trademark), Ethernet (registered trademark), wireless Ethernet) can be coupled to one or more input / output devices such as a keyboard, a pointing device, a scanner, etc., or a network device such as a switch or a router, e.g., via a network adapter.

[0043] Computing device 600 can be implemented in a number of different forms as shown in the figure. For example, it may be implemented as a standard server 666, or multiple times in a group of such servers. It may also be implemented as part of a rack server system 624. Further, it may be implemented in a personal computer such as a laptop computer 622. Or, components from computing device 600 may be combined with other components in a mobile device (not shown) such as device 650. Each of such devices can include one or more of computing devices 600, 650, and the entire system can be composed of multiple computing devices 600, 650 that communicate with each other.

[0044] Computing device 650 includes components such as a processor 652, a memory 664, input / output devices such as a display 654, a communication interface 666, and a transceiver 668. Also, device 650 may be equipped with a storage device such as a microdrive or other device to provide additional storage. Each of components 650, 652, 664, 654, 666 and 668 are interconnected using various buses, and some of the components may be implemented on a common motherboard or in other suitable ways as appropriate.

[0045] Processor 652 can execute instructions within computing device 650, including instructions stored in memory 664. The processor may be implemented as a chipset of chips including separate multiple analog processors and digital processors. The processor may be provided for adjusting other components of device 650, such as, for example, control of the user interface, applications executed by device 650, and wireless communication by device 650.

[0046] ​Processor 652 can communicate with a user via a control interface 658 and a display interface 656 coupled to a display 654. The display 654 may be, for example, a Thin-Film-Transistor Liquid Crystal Display (TFT LCD) or an Organic Light Emitting Diode (OLED) display, or other suitable display technology. Table The display interface 656 may include appropriate circuitry for driving the display 654 to present graphical and other information to the user. The control interface 658 may receive commands from the user and convert the commands for submission to the processor 652. Further, an external interface 662 may be provided in communication with the processor 652 to enable short-range communication with other devices of the device 650. The external interface 662 may enable, for example, wired communication in some implementations or wireless communication in other implementations, and multiple interfaces may be used.

[0047] Memory 664 stores information within computing device 650. Memory 664 can be implemented as one or more of one or more computer-readable media, one or more volatile memory units, or one or more non-volatile memory units. Extended memory 674 can also be provided and connected to device 650 through an expansion interface 672 that can include, for example, a SIMM (Single In Line Memory Module) card interface. Such extended memory 674 may provide additional storage space for device 650 or store applications or other information for device 650. Specifically, extended memory 674 can include instructions for executing or supplementing the above-described processes and can include secure information. Thus, for example, extended memory 674 may be provided as a security module for device 650 and programmed with instructions that enable secure use of device 650. Further, secure applications may be provided via the SIMM card along with additional information such as placing identification information on the SIMM card in a hacking-proof manner.

[0048] The memory can include, for example, flash memory and / or NVRAM memory, as described later. In one implementation, a computer program product is tangibly embodied in an information carrier. The computer program product includes instructions that, when executed, perform one or more of the methods as described above. The information carrier is a computer machine-readable medium or machine-readable medium such as memory 664, extended memory 674, or memory on processor 652 and can be received, for example, via transceiver 668 or external interface 662.

[0049] Device 650 may communicate wirelessly via a communication interface 666 that may include a digital signal processing circuit as needed. The communication interface 666 may enable communication in various modes or protocols such as GSM (registered trademark) voice calls, SMS, EMS, or MMS messaging, CDMA, TDMA, PDC, WCDMA (registered trademark), CDMA600, or GPRS. Such communication may occur, for example, via a radio frequency transceiver 668. Additionally, short-range communication may occur, such as by using Bluetooth, WiFi, or other such transceivers (not shown). Further, a GPS (Global Positioning System) receiver module 670 may provide additional navigation and location-related wireless data that can be used as appropriate by an application running on device 650.

[0050] Also, device 650 can communicate audibly using an audio codec 660, which can receive voice information from a user and convert it into usable digital information. The audio codec 660 can similarly generate audible sound for the user, such as through a speaker within the handset of device 650, for example. Such sound may include sound from a voice call, recorded sound (such as a voice message, music file, etc.), or sound generated by an application operating on device 650.

[0051] Computing device 650 can be implemented in a plurality of different forms as shown. For example, it may be implemented as a mobile phone 680. It may also be implemented as part of a smartphone 682, a personal digital assistant, or other similar mobile device.

[0052] Various implementations of the systems and techniques described herein can be realized in digital electronic circuitry, integrated circuitry, specially designed application specific integrated circuits (ASICs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementations in one or more computer programs executable and / or interpretable on a programmable system including at least one programmable processor coupled to communicate data and instructions between a storage system, at least one input device, and at least one output device, which programmable processor may be dedicated or general purpose.

[0053] These computer programs (also known as programs, software, software applications or code) include machine instructions for a programmable processor and can be implemented in high-level procedural and / or object-oriented programming languages, and / or in assembly / machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus and / or device (e.g., magnetic disks, optical disks, memory, Programmable Logic Device (PLD)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term " "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0054] To enable interaction with a user, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a cathode ray tube (CRT) or a liquid crystal display (LCD) monitor) for displaying information to the user, and a keyboard and a pointing device (e.g., a mouse or a trackball) for enabling user input to the computer. Other types of devices can similarly be used to provide interaction with the user. For example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback). Also, the input from the user can be received in any form including acoustic input, voice input, or tactile input.

[0055] The systems and techniques described herein can be included in a back-end component (e.g., as a data server), or can include a middleware component (e.g., an application server), or can include a front-end component (e.g., a client computer having a graphical user interface or a web browser through which a user can interact with an implementation example of the systems and techniques described herein), or can be realized in any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), and the Internet.

[0056] ​A computing system can include a client and a server. The client and the server are generally separated from each other and typically interact through a communication network. The client-server relationship is created by computer programs that are executed on respective computers and have a client-server relationship with each other.

[0057] Although some embodiments have been described, it will be understood that various changes may be made without departing from the spirit and scope of the invention.

[0058] Also, the logical flow shown in the figures does not require a particular order or sequential order shown to achieve the desired result. Further, other steps may be provided, or steps may be eliminated from the flow described, and other components may be added or removed with respect to the system described.

Claims

1. A processor of a user device executes content on the user device, the user device detects at least one external device within a setting range of the user device, based on an output capability of the at least one external device and an output characteristic of the content executed by the user device, identifies the at least one external device for output of the content executed by the user device, and transfers execution of the content from the user device to the at least one external device for output by the at least one external device. A method realized by a computer, comprising:

2. The method realized by a computer according to claim 1, wherein detecting the at least one external device within the setting range of the user device includes detecting the at least one external device within a visual field of the user device.

3. Identifying the at least one external device includes: detecting a physical feature in image information captured by the user device, identifying a position of the user device based on the detected physical feature, and identifying the at least one external device based on the identified position of the user device. The method realized by a computer according to claim 1 or 2, comprising:

4. The user device includes a handheld device and a head-mounted display device, executing content includes executing content on the handheld device, and detecting a physical feature in the image information includes detecting a physical feature in image information captured within a visual field of the head-mounted display device. The method realized by a computer according to claim 3, comprising:

5. Identifying the position of the user device based on the detected physical feature includes identifying the position of the user device based on a comparison of features pre-stored in a database and accessible to the user device via a server. The method realized by a computer according to claim 3, comprising:

6. Detecting the at least one external device within the setting range of the user device includes detecting the at least one external device as a previously connected device, detecting the at least one external device based on an electromagnetic wave signal, detecting the at least one external device within a shared network, or detecting the at least one external device within a network of a previously connected device, and the method realized by a computer according to any one of claims 1 to 5 includes at least one of them.

7. Detecting the at least one external device within the setting range of the user device includes detecting the at least one external device within the field of view of the user of the user device, and the method realized by a computer according to any one of claims 1 to 5.

8. Transferring the execution of the content from the user device to the at least one external device for output by the at least one external device includes outputting a user verification request before transferring the content from the user device to the at least one external device, receiving a user verification input for verifying the transfer of the content from the user device to the at least one external device, and responding to the reception of the user verification input, transferring the execution of the content from the user device to the at least one external device, and the method realized by a computer according to any one of claims 1 to 7.

9. Outputting the user verification request includes outputting at least one of a visual request or an acoustic request, Receiving the user verification input includes receiving at least one of a touch input or an acoustic input, and the method realized by a computer according to claim 8.

10. A system comprising an output system including an audio output device and an image output device, a sensing system, at least one processor, and a memory storing instructions, and when the instructions are executed by the at least one processor, the instructions cause the at least one processor to Cause the output system to output the content executed by the at least one processor. Cause the sensing system to detect at least one external device within the setting range of the sensing system. Based on the output capabilities of the at least one external device and the output characteristics of the content output by the output system, cause the at least one external device to be identified for output of the content executed by the user device. A system that causes the at least one external device to transfer the execution of the content for output by the at least one external device. When detecting the at least one external device, the instruction causes at least one processor to Detect the at least one external device based on the image information captured by the sensing system. Detect the at least one external device as a previously connected device. Detect the at least one external device based on an electromagnetic wave signal. Detect the at least one external device within a shared network, or The system according to claim 10, which causes at least one of detecting the at least one external device within the network of a previously connected device to be executed. When detecting the at least one external device based on the image information captured by the sensing system, the instruction causes the at least one processor to Detect physical features in the image information. Identify a location based on a comparison between the detected physical features and pre-stored features in a database accessible to the system via a server. The system according to claim 11, which causes the at least one external device to be identified based on the identified location. When transferring the execution of the content to the at least one external device, the instruction causes the at least one processor to Output a user verification request before transferring the content to the at least one external device. Receive a user verification input for verifying the transfer of the content to the at least one external device. ​ ​ ​ The system according to any one of claims 10 to 12, wherein in response to receiving the user verification input, the execution of the content is transferred to the at least one external device.

14. The system according to claim 13, wherein the user verification request includes at least one of a visual request or an acoustic request, and the user verification input includes at least one of a touch input or an acoustic input.

15. The system according to any one of claims 10 to 14, wherein the output system, the sensing system, the at least one processor, and the memory are installed in at least one of a handheld electronic device or a head-mounted display device operable within a network.

Images