Cloud gaming device handover

Abstract

A method and system for identifying options for a secondary client device for device handover of game play includes establishing a game play session for a primary client device by executing the game on a server for streaming video frames to the primary client device. A request is received to generate a configuration file for one or more secondary client devices identified as local to the primary client device. The configuration file is configured to identify device attributes of the secondary client devices. A handover option is provided to the primary client device during game play, the handover option identifying one or more of the secondary client devices based on the configuration file. A selection of a secondary client device identified by the handover option received from the primary client device causes the streaming of video frames to the primary client device to be paused, the game state of the game to be saved, and an option to resume game play on the secondary client device to be provided. A resume request received from the secondary client device causes the game state of the game to be accessed and game play to be resumed, such that streaming of video frames to the secondary client device continues.

Description

[0001] This divisional application is a divisional application with an application date of April 20, 2017, application number 201780033048.4, and invention name “CLOUD GAMING DEVICE HAND-OF-THE-GOOD”. Technical Field

[0002] The present disclosure relates to systems and methods for providing user-interactive gaming, and more particularly, to providing an option for selecting an alternate client device for device handoff during cloud gaming. Background Art

[0003] Related technical description

[0004] One of the rapidly developing technologies is the field of cloud gaming, in which users are able to access multiple games available on a cloud gaming site over a network (such as the Internet) and start playing games. The user accesses their account on the cloud gaming site and selects a game for game play from a list of games available to the user account. When the user selects a game for game play from the cloud gaming site using a client device, the server in the cloud gaming site starts a game play session and starts streaming the video frames of the game to the client device that the user accessed the cloud gaming site. During the session, if the user wants to switch to a second client device to play the game, the server on the cloud gaming site must restart the game from the beginning so that it can start streaming the video frames to the second client device. Before the server can start streaming the video frames of the game to the second client device, the server must perform bandwidth testing and other quality of service tests. The restart of the game and the pre-testing of the second client device cause the user to wait unnecessarily, which frustrates the user.

[0005] It is in this context that the embodiments of the present invention come into being. Summary of the Invention

[0006] Embodiments of the present invention disclose methods, systems, and computer-readable media for identifying options for alternate client devices for handing over game play of a game hosted by a cloud gaming site. Various embodiments enable a primary client device currently used to play a game hosted by a cloud gaming system on a cloud gaming site to participate in background profiling of secondary client devices that are identified as local to the primary client device. Profiling of the secondary client devices is performed to determine which of the secondary client devices are eligible to continue the game play session of the game, and the profiling of the secondary client devices is performed before a device handover request is received from the primary client device. The profile identifies device attributes of each of the secondary client devices. In some embodiments, the cloud gaming system may prepare different secondary client devices for gaming using a profile for preconfiguring services to the secondary client devices. Each game hosted by the cloud gaming system may have its own service and attribute requirements.

[0007] Profiling allows the cloud gaming system to identify the services and device attributes available in each of the secondary client devices and to determine which services and device attributes are unavailable or not at the level required for the respective secondary client devices to interact with the game executed on the cloud gaming site. In some embodiments, the cloud gaming system can use a configuration file for each secondary client device to pre-configure appropriate resources to enhance the services or device attributes of specific ones of the secondary client devices. Such pre-configuration can be performed by the cloud gaming system in advance or in the background during a game play session initiated by a primary client device, with the expectation that a specific one of the secondary client devices can be selected by the primary client device for device handover for game play during the game play session.

[0008] When a request for device handoff is received from a primary client device at the cloud gaming system, profiling allows the cloud gaming system to perform a faster, frictionless device handoff from the primary client device to a specific secondary client device, and the secondary client device will have the necessary resources (services, device attributes) to continue the game play session. In other words, profiling allows the cloud gaming system to define the secondary client device so that when a handoff request is received, the system can switch the client device used by the user to interact with the cloud gaming site for game play.

[0009] A successful device change will include the server of the cloud gaming system executing the game pausing the game play and stopping the streaming of video frames of the game to the primary client device, saving the game state of the game, and providing an option to resume the game play of the game to the secondary client device identified by the handover option. Upon receiving the resume request from the secondary client device, the server on the cloud gaming system retrieves the game state, resumes the game play from where it was paused, and continues streaming video frames for the game to the secondary client device.

[0010] Discovering and profiling secondary client devices in advance saves considerable wait time during device handover. Secondary client devices identified in the handover options are qualified to provide a gaming experience comparable to that experienced using the primary client device. Furthermore, profiling and pre-configuration ensure that video frames are provided to the user at the secondary client device with minimal latency. Pausing the game and / or saving the game state during device handover allows for faster resumption of gameplay.

[0011] In one embodiment, a method is provided. The method is executed by a server of a cloud gaming system and is used to identify options for a secondary client device that can be used to handover a game play device to one of the secondary client devices. The method includes, in response to a request to play a game received from a primary client device, establishing, by the server, a game play session for the primary client device. The server of the cloud gaming system is configured to play the game and stream video frames to the primary client device. The server receives a request to generate a configuration file for one or more secondary client devices identified as local to the primary client device. The configuration file is configured to identify device properties of the secondary client devices. During game play, the server provides handover options to the primary client device. The handover options are configured to identify one or more of the secondary client devices for device handover based on the configuration files of the secondary client devices. A selection of a specific secondary client device identified by the handover options is received from the primary client device. The selection is configured to pause the streaming of video frames to the primary client device and save the game state of the game. The pause is configured to provide an option to resume game play on the specific secondary client device. A resume request is received from the specific auxiliary client device to continue the play of the game on the auxiliary client device. The resume request is configured to cause the server to access the game state of the game and continue streaming video frames to the specific auxiliary client device.

[0012] In some embodiments, the request to generate the configuration file is received during the session.

[0013] In some embodiments, the particular secondary client device is designated as the primary client device for the remainder of the session.

[0014] In some embodiments, the request to generate the configuration file is triggered in response to the occurrence of a triggering event.

[0015] In some embodiments, the request for the configuration file is received from each of the secondary client devices identified as local to the primary client device.

[0016] In some embodiments, the device attributes used to define the auxiliary client devices are ranked according to the requirements of the game. The auxiliary client devices provided in the handover options are organized according to the ranking of the device attributes.

[0017] In some embodiments, the secondary client device is identified using one or more service discovery protocols implemented within the primary client device and the secondary client device.

[0018] In some embodiments, the request to generate the configuration file is received from the primary client device. The request includes a device identifier of the secondary client device for which the configuration file is to be generated.

[0019] In another embodiment, a method is performed by a server of a cloud gaming system. The method is performed to identify options for handing over a game play device to one of the auxiliary client devices. The method includes establishing a game play session for the primary client device in response to a request to play the game from a primary client device. The server of the cloud gaming system is configured to play the game and stream video frames to the primary client device. One or more auxiliary client devices identified as local to the primary client device are detected. A configuration file is generated for the primary client device and one or more of the auxiliary client devices identified as local to the primary client device. The configuration file is configured to identify device attributes of each of the primary client device and the auxiliary client devices. During the game play, the server provides handover options to the primary client device. The handover options are configured to identify one or more auxiliary client devices, based on the configuration files generated for the primary client device and the auxiliary client devices, the one or more auxiliary client devices being local to the primary client device and having device attributes that can enhance the user's gaming experience. A selection of a specific auxiliary client device identified by the handover options is received by the server from the primary client device. The selection is configured to pause the streaming of video frames to the primary client device and save the game state of the game. The pause is configured to provide a resume option for resuming play of the game on the specific secondary client device identified by the handover option. A resume request is received by the server from the specific secondary client device to continue the game play session on the secondary client device. The resume request is configured to cause the server to retrieve the game state of the game and continue streaming video frames to the specific secondary client device.

[0020] In some embodiments, the configuration files for the primary client device and the one or more secondary client devices are generated in response to a triggering event.

[0021] In some embodiments, the configuration files for the primary client device and the one or more secondary client devices are generated in response to a request from the primary client device.

[0022] In another embodiment, a cloud gaming system is disclosed. The cloud gaming system includes an application server configured to execute multiple games. A game execution engine in the application server receives a request to play a game from a primary client device. In response, the game execution engine executes an instance of the game and generates video frames of the game. The application server is configured to process the generated video frames, encode the video frames, and stream the video frames to the primary client device. The client device profiling module within the application server is configured to receive a request to generate a configuration file for one or more auxiliary client devices identified as local to the primary client device, wherein the one or more auxiliary client devices are identified based on the configuration file of the primary client device. The client device profiling module is further configured to generate a list of the auxiliary client devices. The handover manager within the application server is configured to obtain a list of the auxiliary client devices, define the auxiliary client devices for use in playing the game, and forward a refined list of qualified auxiliary client devices to the primary client device with a handover option. The handover manager is further configured to receive a selection of a specific secondary client device identified by the handover option and, in response, forward a signal to the game execution engine to pause the streaming of the video frames to the primary client device. In some embodiments, in response to the pause signal, a game state of the game is also saved. When the handover manager receives a resume request from the specific secondary client device, in response, the handover manager signals the game execution engine to resume game play and continue streaming the video frames to the specific secondary client device. In embodiments where the game state of the game is saved, in response to the resume request, the game execution engine may retrieve the game state of the game and resume game play of the game from the point at which the game was paused, as indicated by the game state of the game.

[0023] In some embodiments, the handover manager is configured to, in response to receiving a selection of the particular secondary client device identified by the handover option, signal a pause manager to pause the game play of the game. The pause manager is configured to forward a signal to the game execution engine to pause the game play of the game.

[0024] In some embodiments, the handoff manager is configured to signal an operating system of the application server to pause transmission of video frames of the game to the particular primary client device in response to selection of the particular secondary client device identified by the handoff option.

[0025] In some embodiments, the handover manager is configured to, in response to receiving a resume request from the particular secondary client device, signal a resumption manager to resume the game play of the game. The resumption manager is configured to forward a signal to the game execution engine requesting the game execution engine to retrieve the game state of the game and resume streaming video frames of the game to the particular secondary client device.

[0026] In some embodiments, the handover manager is configured to signal a resume manager to resume game play. The resume manager receives the signal and, in response to receiving a resume request from the specific secondary client device, signals the operating system of the application server to resume transmitting video frames of the game to the specific secondary client device.

[0027] In some embodiments, the handover manager is further configured to designate the particular secondary client device as the primary client device.

[0028] Other aspects and advantages of the present invention will become apparent from the following detailed description read in conjunction with the accompanying drawings, which illustrate the principles of the present invention by way of example. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The invention, together with further advantages thereof, may best be understood by referring to the following description taken in conjunction with the accompanying drawings.

[0030] Figure 1 A simplified block diagram illustrating an exemplary system for identifying options for a secondary client device for device handoff for game play according to an embodiment of the present invention.

[0031] Figure 2 An exemplary application server having various modules for generating handover options identifying one or more secondary client devices for device handover is shown in accordance with an embodiment of the present invention.

[0032] Figure 3 A master client device is shown with a switching module for initiating device handover according to one embodiment of the present invention.

[0033] Figure 4 An exemplary discovery agent of a handoff module in a primary client device for discovering secondary client devices local to the primary client device is shown according to an embodiment of the present invention.

[0034] Figure 5 An exemplary client device profiler for a server (such as an application server) in a cloud gaming system according to an embodiment of the present invention is shown for profiling secondary client devices.

[0035] Figure 6A An exemplary configuration file generated for a secondary client device local to a primary client device according to an alternative embodiment of the present invention is shown.

[0036] Figure 6B 、 Figure 6C-1 and Figure 6C-2 An example of handover logic at a primary client device providing input for selecting a secondary client device for device handover is shown in accordance with an embodiment of the present invention.

[0037] Figure 7 An exemplary operational flow is shown for a method for identifying options for a secondary client device for device handoff for game play, according to an embodiment of the present invention.

[0038] Figure 8 An exemplary information service provider architecture for delivering information content and services to geographically dispersed users connected via a network is shown in accordance with one embodiment of the present invention.

[0039] Figure 9 A simplified block diagram of an exemplary gaming system is shown according to various embodiments of the present invention. DETAILED DESCRIPTION

[0040] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced without some or all of these specific details. In other cases, well-known process steps have not been described in detail in order to avoid obscuring the present invention.

[0041] According to various embodiments of the present invention, a user can use a master client device to access the cloud gaming system through a user account in order to select a game for game play. The cloud gaming system is configured to use the resources available to the cloud gaming system to host multiple online games for game play. In response to the user accessing the cloud gaming system, the cloud gaming system uses the user account information stored in the user account database to authenticate the user account and provides a list of games available to the user account for game play. The user's selection of a game for game play triggers the cloud gaming system to identify a data center located near the geographic location of the master client device and sends a request to the server in the data center to execute an instance of the game and establish a game play session. The server provides the necessary resources (i.e., processing and communication resources) to execute the game, encode the video frames of the game, and start streaming the encoded video frames of the game to the master client device for rendering and user interaction. The user interaction provided at the master client device is used to affect the game outcome of the game executed on the cloud gaming system. In some embodiments, shortly after initiating the game play session with the master client device, the server of the cloud gaming system may generate a configuration file for the master client device. Profiling of the primary client device can be initiated as part of a quality of service (QOS) validation to ensure that bandwidth, latency, and other resource variables are adequate for the primary client device, as well as as part of data center selection and qualification so that the primary client device can receive and process streaming video frames of the game streamed from the server at the data center. It should be noted that the various embodiments discussed herein can also be extended to the generation of audio frames, encoding of audio frames, and streaming of encoded audio frames for the game to the primary client device for rendering.

[0042] During gameplay, the primary client device may actively seek and discover secondary client devices local to the primary client device, and send a signal to each of the secondary client devices, or send a signal with a request to be parsed by the cloud gaming system's server, to select some of the secondary client devices. The configuration file may be used to define the secondary client devices for game play. The request generated by the primary client device identifies the server that the secondary client device must use for parsing. The server selected for parsing may be a server in the data center currently executing the game, or any other server configured to perform such parsing within the same data center or any other data center within the cloud gaming system. The cloud gaming system's parsing server includes configuration file generation logic for generating a configuration file for each client device upon request. In some embodiments, when the primary client device discovers a secondary client device, the discovered secondary client device is parsed. A handover option is set at the primary client device, where the handover option identifies one or more secondary client devices eligible for device handover based on the generated configuration file for the secondary client device. In some embodiments, the handover option may be set to handover streaming of only video frames, only audio frames, or both audio and video frames of the game. When the user selects a specific secondary client device, the user selection is forwarded to the server, which generates a pause signal. In response to receiving the pause signal, the server in the data center pauses the game play and provides a resume option to the specific secondary client device to resume the game play.

[0043] As part of processing the pause signal, in some embodiments, the server executing the game at the data center can start an event trigger, which causes the server to save the system state when the event trigger is started. In this type of embodiment, the system retains all applications (including game applications) and / or system states without directly manipulating any game attributes or game variables. The system state can be used to recreate the game state of the game. In some embodiments, an application programming interface (API) can be used to save the system state. In an optional embodiment, in response to the pause signal, the server can write out the saved data of the game when receiving the pause signal. The saved data records the progress achieved by the user during the game and can then be used to recreate the current game state of the game when the game resumes. In one embodiment, for more information about how to use the progress in the user's game to recreate the game state of the game, reference can be made to U.S. application number 12 / 917,388, filed on November 1, 2010 and entitled "USER INTERFACE, SYSTEM AND METHOD FOR CONTROLLING AVIDEOSTREAM," the entire contents of which are incorporated herein by reference.

[0044] In some embodiments, a user's selection of a specific secondary client device indicates that the user no longer wishes to provide game interaction for the game from the primary client device. In some embodiments, a user's selection of a specific secondary client device may indicate that the user no longer wishes to receive at least a portion of the streamed game content (i.e., audio or video frames of the game content) at the primary client device. For example, a user may not wish to receive the audio portion of the game content at the primary client device, but may still be interested in receiving the video portion of the game content at the primary client device. This may occur, for example, if the audio reception at the primary client device is not at an acceptable level. In such embodiments, the audio portion of the game content may be extracted and forwarded to the specific secondary client device, while the video portion of the game content (e.g., where the audio portion is muted) may continue to be streamed to the primary client device. In such embodiments, the handover options provided at the primary client device may include an option to select an appropriate portion of the streamed game content (e.g., video, audio, etc.) and a specific secondary client device for streaming the selected portion of the game content as part of the device handover, such that the selected portion of the streamed game content (audio or video frames) is redirected to the specific secondary client device.

[0045] In some embodiments, in response to receiving a pause signal, a server in a data center may pause the transmission of streaming video frames and / or streaming audio frames for the game to the primary client device. As a result, the game appears to be in a "paused" state. In this embodiment, the game may continue to be active but will not receive any user interaction until the device handover is complete.

[0046] The selection of the recovery option at specific auxiliary client device is forwarded to the server, thereby causes the server to access the game state (if any) of the game and / or resume the game from the point where the game was paused. The game that resumes is carried out so that the video frame and / or the audio frame of the game are streamed to specific auxiliary client device. Alternatively, when the transmission of the paused video frame and / or audio frame keeps the game active, the recovery request will cause the server to resume the streaming video frame and / or streaming audio frame of the game to specific auxiliary client device. In some embodiments, based on the user interaction received from specific auxiliary client device, video frame and / or audio frame are generated and streamed. In the embodiment where a part of game content is streamed to main client device and remainder is streamed to auxiliary client device, based on the user interaction received from specific auxiliary client device, main client device or from main client device and auxiliary client device, video frame and / or audio frame are generated and streamed to appropriate client device (that is, main client device and auxiliary client device).

[0047] The auxiliary client devices identified in the list have device attributes that are required for gameplay of the game and, in some embodiments, are available to the game at the time the handover option is received. Various embodiments discussed herein provide a user with the option to switch to an auxiliary client device for gameplay during a gameplay session, and this switching is accomplished in a seamless and frictionless manner. The auxiliary client devices identified in the options can be discovered using any of a number of device discovery protocols implemented in the client device, based on signals generated by the auxiliary client device, based on other devices receiving signals from the auxiliary client device, etc. The discovered client devices are profiled in the background and qualified for gameplay of the game, so that device handover can be successfully performed.

[0048] The information provided in the configuration file of the auxiliary client device can be used not only to determine the device attributes required for gameplay of the game, but also to determine device attributes that are not available or not at the level required for gameplay of the game at one or more auxiliary client devices. Based on this configuration file information, the cloud gaming system can, in some embodiments, pre-configure services and / or device attributes in one or more auxiliary client devices and prepare them for gameplay so that when the user of the primary client device selects the handover option, the switching of client devices is completed in a quick and frictionless manner. In some embodiments, device profiling can be initiated in response to the occurrence of a triggering event, such as a user action, a calendar event, a signal from the primary client device (e.g., a low-power signal, etc.), a social media event, etc., in response to a change in one or more device attributes of the primary client device.

[0049] The secondary client devices may be part of the same network as the primary client device. Once the secondary client devices have been discovered, the primary client device may prompt each of the discovered secondary client devices by sending a signal to initiate a connection with a server (such as an application server in a cloud gaming site) and exchange information with the server. In other embodiments, once the secondary client devices have been discovered, the primary client device may prompt the server of the cloud gaming system by sending a signal to profile one or more secondary client devices in order to initiate a connection with each of the discovered secondary client devices and exchange information with the secondary client devices. The information that may be exchanged between the secondary client devices and the server includes, for example, session information, quality of service information, device capabilities, games in progress or available for play, video codec information, audio codec information (if available), speaker configuration, device availability, hardware properties, software properties, and the like.

[0050] Because each game may have different resource requirements, the device attributes required to initiate a streaming session for each game may differ. Consequently, the information provided in the configuration file can be used to evaluate the device capabilities of each discovered secondary client device to determine whether the corresponding secondary client device possesses sufficient device attributes to initiate a streaming session for the game. For example, as part of this evaluation, the network bandwidth of each secondary client device may be monitored continuously, periodically, or at predefined times. The results of this monitoring can be used to update the configuration file for the corresponding secondary client device to maintain the current device attributes of the secondary client device. Furthermore, each of the primary and secondary client devices may have different types of inputs for providing game interaction. For example, client device A may use a gamepad, client B may use a touchscreen, and client C may use a keyboard to provide game interaction for a game configured to receive user interaction from such inputs. Consequently, in some embodiments, the information provided in the configuration files of the discovered secondary client devices can be used to identify the type of input used by each client device to provide input to the game, thereby adjusting the input method when a device handover occurs. For example, the primary client device may use a touch screen input, and the selected secondary client device may use a game controller, with both input methods being acceptable for receiving interactions for the game. For example, when a device handover occurs, the switching of the client device from the primary client device to the selected secondary client device will cause the game to recognize the difference in input methods between the two client devices (i.e., the primary client device and the selected secondary client device) and switch the input method used to receive game interactions for the game from touch screen input to game controller input. In some embodiments, the game may not be able to recognize the input method used by the selected secondary client device. In such embodiments, the cloud gaming system or the selected secondary client device may provide input emulation (e.g., an on-screen virtual input method such as a virtual game controller or a virtual keyboard, a virtual numeric keypad, virtual game controls, etc.) at the display portion of the secondary client device, and the type of input emulation selected for display may be based on the type of client device. Similarly, each client device has its own resource capabilities and limitations (e.g., screen / display resolution, aspect ratio, audio capabilities, etc.), and these are reflected in the profile information of the corresponding client device. The information provided in the device profile can be used to qualify client devices, define input methods for the game, and / or adjust streaming video frames.

[0051] Based on the device profile of the secondary client device, the user of the primary client device may decide to initiate a device handoff to a specific one of the discovered secondary client devices. The decision to initiate a device handoff may be based on a determination that the secondary client device has device attributes that may provide the user with a better gaming experience, such as a higher resolution, a higher frame rate, lower latency, etc. Alternatively, the decision to initiate a device handoff may be based on the fact that the primary client device is no longer able to support gameplay of the game due to a lack of power or unavailability of one or more device attributes of the primary client device.

[0052] As part of the device handover, a device handover option and a list of one or more auxiliary client devices that are predefined to provide optional, comparable or better gaming experiences are provided for user selection at the primary client device. The user's selection from the list at the handover option or at a specific auxiliary client device triggers the device handover. Since the server has predefined the auxiliary client devices included in the list, the user's selection will result in a faster, efficient, frictionless and seamless device switch, so that the server can continue to stream video frames for the game to the selected auxiliary client device. In some embodiments, in addition to switching the streaming of video frames and / or audio frames from the primary client device to the selected auxiliary client device, the switch may also include switching one or more controllers for providing input to the game. Typically, the user can use a wireless controller (e.g., based on Bluetooth or other wireless technology) associated with the primary client device to interact with the game. In such embodiments, when the device handover is triggered, the switch may also include handing over the Bluetooth connection of the wireless controller from the primary client device to the selected auxiliary client device. As part of the device handover, the primary client device may notify the secondary client device of the properties of the wireless controller used to provide input to the game. In addition to notifying the secondary client device, the primary client device may disconnect the wireless controller. The selected secondary client device receives the notification and may use the device properties of the wireless controller to automatically discover the wireless controller and form an association with the wireless controller, so that the selected secondary client device can use the newly associated wireless controller to provide input. In an alternative embodiment, a wireless controller with similar device properties may already be associated with the selected secondary client device. In such an embodiment, the secondary client device may not undergo a discovery process for discovering the wireless controller or an association process to associate the wireless controller with the primary client device. Instead, once it is determined that the wireless controller has the necessary device properties for providing input to the game, the wireless controller already associated with the selected secondary client device can be used to provide input to the game, and the game will receive and map the input received from the wireless controller to update the game state of the game.

[0053] In some embodiments, device discovery can be initiated by the primary client device when the primary client device is idle, or can be initiated during a game play session, periodically, or when running different applications. Profiling of the secondary client devices is performed in the background to obtain the quality of service and other status of the secondary client devices, so that when and if the user decides to change the device used for gaming, device switching can easily occur because the device properties and device settings are pre-established for fast stream switching. With the invention generally understood, specific embodiments will now be described with reference to the various drawings.

[0054] Figure 1 A simplified block diagram of a system for identifying options for secondary client devices that can be used for device handover for game play in one embodiment of the present invention is shown. The system includes multiple client devices 100-1 to 100-n, which are communicatively connected to a server (320), such as an application server, at a data center 310 through a user account of a cloud gaming system 300. The client devices 100 access the server 320 via a network 200, such as the Internet.

[0055] The client device 100 (i.e., any one of 100-1 to 100-n) is any computing device that includes a processor, a memory, a network connection to the network 200, an appropriate API for communicating with the server-side application, and one or more decoders for decoding the content provided by the server-side application (such as a game application). The processor is capable of executing the client-side application, which can interact with the server-side application by connecting to the network via a network connection and using an application programming interface (API) to communicate with or access the server-side application. The network connection can be a wired or wireless connection. The client device 100 can be a thin client, a general-purpose computer, a dedicated computer, a game console, a personal computer, a laptop computer, a tablet computing device, a mobile computing device, a portable gaming device, a cellular phone, a smart phone, a head-mounted display, a smart wearable device, a set-top box, a streaming interface / device, a smart TV or networked display, or any other computing device that can be used to access applications available on a remote server. The network connection and communication protocol available at the client device 100 enable the client device 100 to communicate with a remote server to receive content, including streaming video frames of multimedia content, from the remote server (such as a server that is part of the cloud gaming system 300). The video frames representing the game play content streamed by the remote server have been compressed at the remote server using an encoder before being streamed to the client device. The client device 100 may include a decoder for decompressing the streaming video frames and rendering the content using corresponding components of the client device 100.

[0056] In some embodiments, the amount of processing performed by the client device 100 may vary relative to input and output processing. Broadly speaking, the game or application is primarily maintained, executed, and compressed / encoded on a game server or other application server 320 available within the data center 310, with the client device 100 primarily responsible for receiving, decoding, processing, and rendering audio / video data on the display of the client device 100. The client device 100 is further configured to transmit user input back to the game server or other application server. The client device 100 can be a standalone computing device connected to a display for rendering video data. In other embodiments, the display can be integrated into the client device 100. In one embodiment, the display is a networked display device that utilizes the display device's network connection to provide a platform operating system for the application or "app." In such an embodiment, the client device 100 can be defined by an application executing on the platform provided by the display device's operating system.

[0057] During a game play session, client devices 100-1 through 100-n may actively seek one or more other client devices local to them. The client device 100 used to initiate a game play request is referred to herein as a "primary" client device, and client devices local to the primary client device and discovered for device handoff are referred to as "secondary" client devices. Depending on the location of the primary client device 100, the secondary client devices detected as local to the primary client device may vary. For example, secondary client devices identified as local to the primary client device 100 in a user's home network may differ from secondary client devices identified in the user's friend's home network, which may differ from secondary client devices identified in an office environment, and so on.

[0058] The server used in this application can be a remote server, a virtual computer, a cloud gaming server, a cloud application server, a remote application server, a digital media server, a server for providing a game developer / game sponsor storefront, a website server, a terminal server, a console server, or any other type or form of server computing device available in a data center that can host one or more games or applications that users can access and interact with during cloud gaming (including providing or allocating processing resources for executing the games or applications). The server may include an encoder that compresses data in video frames and forwards the compressed video frames in a data stream to the client device 100 using an application programming interface (API) call that follows a specific type of protocol.

[0059] For example, server 320 (such as a cloud gaming server in data center 310) executes a video game selected by a user for game play, defines the state of the video game from time to time, and sends streaming video frames (including image, video, and audio data) to primary client device 100, initiating a request for game play of the game at primary client device 100. In turn, user input module 107 of primary client device 100 is configured to process user input from the user playing the video game and transmit the input data to server 320. Server 320 is configured to process the input data to affect the game state of the video game.

[0060] The cloud gaming system includes multiple data centers 310-1 to 310-m. In one embodiment, the data center 310 may include multiple servers 320 (e.g., game servers), a storage system capable of storing game code, application code, user-related and application-related data storage areas, and may make them easily accessible so as to be able to handle various requests from multiple users. The data center may also include telecommunications equipment (such as routers, switches, etc.) for establishing communication connections between the client device 100 and the multiple servers 320. Each of the multiple servers 320 may be equipped with a server-side API (different or similar) for communicating with a corresponding client-side API at the client device 100, or with a server-side API associated with a third-party social media provider. In some embodiments, the servers 320 in the data center 310 may be configured to execute various types of applications, including game applications, and stream application content (i.e., video frames generated by the game application) to the corresponding client device 100 for rendering. The server 320 can be configured to use any number of compression techniques to perform compression operations on any data generated or provided by the server 320, and use any one of the communication protocols and / or transmission protocols to forward the compressed data to the client device 100. The server may include a terminal server, console server, virtual server, etc. that are generally used to perform or implement specific functions, games, or applications. To name a few, some examples of the functions, games, or applications performed by the server may include database management, file management, mail services, print services, web services, game management, application management, media management, directory services, communication management, computing services, and agent management. In some embodiments, the server 320 (such as a console server) can simulate a game console by performing a game and providing streaming video frames for rendering to one or more client devices 100. In some embodiments, multiple servers and / or storage devices can be provided as rack servers or storage devices, wherein each data center includes multiple rows of servers and / or storage racks. Each server may be able to perform multiple applications and / or provide a variety of services.

[0061] The server 320 may also include a device profiler that can be used to profile client devices. The device profiler can be configured to establish a communication connection with one or more client devices to exchange information with the client devices. The exchanged information is used to identify device attributes and generate a profile for each client device.

[0062] It should be appreciated that the cloud gaming system 300 facilitates single-player or multi-player cloud-based gaming from players located in different geographical locations by allowing one or more instances of a video game to be executed by one or more servers 320 (e.g., application or game servers) located in one or more data centers, which can be accessed by the players via the network 200. In this way, the execution of the video game is not dependent on the hardware or network conductivity of any single player, which would affect the user experience of that given player.

[0063] The operations performed using the cloud gaming architecture described herein form technical operations that require multiple servers and / or execution platforms to enable rapid access to databases and presentation of content (such as game content) to remotely located user's client devices 100. Cloud gaming may also include compression operations performed by the cloud gaming server using any number of compression techniques. The compression techniques may use standard encoders, which will then allow standard decoders on the client devices 100 to access and decode the data. In some embodiments, dedicated encoders and decoders may also be implemented in the server and client devices, respectively, to allow specialized encoding and decoding. The decoded data may be further processed in the client device to identify image, video, and audio data, which are then presented using appropriate device components on the client device 100 to enable video game play.

[0064] The management and distribution of video games may include a number of data centers, targeting servers, quality of service testers or engines, targeting and redirection to data centers with lower latency. It should also be understood that these operations and tasks will utilize dedicated server computers designed to stream and reduce latency caused by remotely executing games, as well as deliver streaming game or application data to client devices 100.

[0065] When a request to access the cloud gaming system 300 is received from the master client device 100, a server on the application hosting system (such as the cloud gaming system 300) interacts with the user account data processing module 302 to obtain user-related information. The user account data processing module 302 queries and receives user account information from the user account database 312 that stores user profiles and other user account information. The user profile and user account information are used to identify the user associated with the master client device 100 that initiated the request, as well as to authenticate the user request. After user authentication, the client request is serviced by the server associated with the cloud gaming system 300. In one embodiment, the server of the cloud gaming system 300 can determine all games and / or applications available to the user account, including purchased games / applications, games / applications that the user of the user account is authorized to view and play, and all games and / or applications that are available for free. The game title selector module 304 retrieves game titles for the user account and returns the game titles in the list for display on the user interface at the display of the master client device 100. In one embodiment, user selection of any one of the game titles rendered on the client device is detected, and a signal is sent from the master client device 100 to a server of the cloud gaming system 300 notifying the server of the user selection of the game title.

[0066] The cloud gaming system 300 hosts multiple applications, including gaming applications. In response to receiving a user selection of a game title, the server of the cloud gaming system 300 may identify a data center 310 to service a game play request for the selected game title and forward the request to the data center 310 to service the request. For example, a data center 310 may be selected based on the geographic location of the primary client device 100. Selecting a data center 310 to service a request based on geographic location may have its own drawbacks. For example, due to high demand for its resources, a data center 310 in a geographic location may experience network congestion. Alternatively, a data center 310 in a geographic location may be unable to service a game play request from a particular primary client device due to a lack of a peering agreement between the network provider serving the primary client device and the network provider associated with the data center's operations. The aforementioned drawbacks are merely examples, and other drawbacks may exist. To overcome such drawbacks, in another example, a data center 310 may be selected based on the results of network performance tests, such as bandwidth / latency tests. For example, when a primary client device initiates game title selection for game play, the server of the cloud gaming system 300 may determine the geographic location associated with the primary client device and identify one or more data centers 310 to service the request. When assigning data centers, the cloud gaming system 300 may recognize that servers within a data center 310 located in the primary client device's geographic location may not have sufficient resources to service the request. Consequently, the cloud gaming system 300 may identify a data center 310 located outside the geographic location where the primary client device is based to service the request. For example, based on bandwidth testing, a data center located outside the geographic location may be identified over other data centers in the geographic location associated with the primary client device.

[0067] As previously described, the data center 310 may include a plurality of servers 320, wherein each server 320 executes or is configured to execute one or more game titles available at the data center 310. The game title data store 314 in each data center 310 provides, at one or more servers 320 in the data center 310, a list of game titles available at the data center 310 and the necessary game code for executing the game identified by the game title.

[0068] The server 320 in the data center 310 identified for servicing the game play request establishes a game play session by executing an instance of the game, generates video frames for the game, and streams the video frames to the primary client device 100 for rendering. The server receives user interactions provided at the primary client device 100 and uses them to influence the outcome of the game and update the game state. The client device can be used to identify other client devices in its local area. The identified other client devices are provided as options for device handover.

[0069] Figure 2 In one embodiment, various modules of a server-side application 321 executing on a server 320 within a data center 310 of a cloud gaming system 300 are shown for identifying options for an auxiliary client device for device handoff during game play. Figure 2 In one embodiment shown, the server-side application 321 may include a module of a gaming application 321-a integrated with a module of the handover application module 355. In some embodiments, the module of the gaming application 321-a is used to execute the game and provide streaming video frames of the game to a client device, such as the master client device 100, during the game. In such embodiments, the video frames are encoded before being transmitted to the master client device. In some embodiments, an application programming interface (API) that is part of the gaming application may be used to transmit the video frames. The API may include logic that captures the video frames generated by the gaming application and encodes the video frames using the encoding technology available therein and transmits the encoded video frames to a client device, such as the master client device 100. It should be noted that the various embodiments discussed herein for processing video frames may also be extended to the processing of audio frames.

[0070] In some embodiments, the encoding of video frames can be based on the 'region of interest' provided by the game. Typically, when encoding game data frames, the encoder must comply with the 'bit budget'. The bit budget defines how bits are used to encode the frame in order to provide the best data content. Depending on the bit budget, the encoder attempts to allocate more bits to certain areas of the frame (e.g., areas with dense content of the frame) while allocating fewer bits to other areas of the frame (e.g., areas with less content). In order to help the encoder optimally encode video frames, the game provides information related to the region of interest so that the encoder can identify areas that should receive more details (i.e., areas with text, dense game scenes (such as grass, leaves, forests, etc.), dense action, etc.) and areas that should receive less details (i.e., less obvious areas in the frame). Equipped with this level of refinement of the game and the bit budget, the encoder allocates bits to the video frame in order to provide the best frame content for viewing.

[0071] In an optional embodiment, the streaming video frames of the game generated during the game are forwarded to the operating system of the server 320 or to a graphics processing unit (GPU) driver that interacts with the game application 321-a and / or the operating system. The GPU can process the video frames and provide them to the operating system of the server 320. The operating system of the server 320 can receive the processed video frames and perform some form of encoding in the background before transmitting some form of encoding to the main client device 100. In some embodiments, before sending the video frames to the main client device 100 for rendering, the operating system of the server 320 in the cloud gaming system 300 can use the encoder available in the cloud gaming system 300 to perform encoding of the video frames. As in the previous embodiment, the encoder can receive detailed information about the area of ​​interest of the game and encode the video frames by allocating the data bits that constitute the video frames according to the detailed information. The embodiments discussed here with reference to the processing of video frames can also be extended to audio frames. Before forwarding the video frames and / or audio frames to the client device, the operating system can monitor the network conditions and responsively adjust the frame rate, bit rate, etc. of the video frames and / or audio frames.

[0072] In an alternative embodiment, an external application located outside of the cloud gaming system 300 may capture a High-Definition Multimedia Interface (HDMI) video signal representing a streaming video frame transmitted by the operating system of the cloud gaming system 300 and may use an external encoder to encode the video signal. The external application then processes the encoded video signal for transmission to the client device. As part of the transmission, the external application may monitor network conditions and adjust the quality (i.e., frame rate, bit rate, etc.) of the video frames being streamed to the host client device 100.

[0073] In some embodiments, the operating system or external application can automatically trigger a device handover based on network conditions. As part of the triggering, the operating system or external application can discover other auxiliary client devices located in the vicinity of the primary client device, signal the discovered auxiliary client devices to be profiled by the server, and forward the auxiliary client device information to the handover application module 355 for use during the device handover.

[0074] The handover application module 355 includes a module that identifies auxiliary client devices local to the primary client device, or uses auxiliary client device information provided by the operating system or an external application to profile the auxiliary client device and present it as an option for device handover. In other embodiments, the server-side application 321 may include a game application 321-a separate from the handover application module 355. In such embodiments, the game application 321-a and the handover application module 355 may execute separately on the same server, or the game application 321-a may execute on a first server, while the handover application module 355 executes on a second server in a data center, with the first and second servers communicatively coupled to each other to exchange data, such as user selections, game status, pause / resume requests, and the like. In some embodiments, the game application 321-a that generates streaming video and / or audio frames may have the ability to monitor the network connection to the primary client device and share this information with the handover application module 355. Optionally, network connection quality information may be provided to the handover application module 355 by the server's operating system or an external application.

[0075] The handover application module 355 can use the network connection information of the primary client device and the configuration files generated for the auxiliary client devices to determine whether one or more auxiliary client devices local to the primary client device have a better network connection quality than the one available at the primary client device. When the handover application module 355 identifies one or more auxiliary client devices, it can generate an information message with the option to switch to an alternate auxiliary client device to improve the gaming experience. During device handover, the information message and the list of alternate auxiliary client devices can be forwarded to the primary client device for rendering on the client side for user selection. In some embodiments, the information message can be provided as an overlay, rendered on top of the game scene rendered on the display portion of the primary client device. Alternatively, the information message can be rendered to the server side by the game application 321-a, a server-side process, or a user interface of a server-side game console. In some embodiments, the user interface of the server-side game console is hidden but renders notifications in response to trigger actions, such as receiving a new email, receiving a new message, detecting an online user or a user's friend, etc., where the trigger actions can be expanded to also include receiving an information message.

[0076] The functions or effects of various modules will refer to Figure 2The server-side application model shown in FIG3 is explained in terms of the integrated server-side application model, but is extensible to the other server-side application models discussed above. As previously described, the server-side application 321 includes modules of the game application 321-a integrated with modules of the handover application module 355. The multiple modules of the game application 321-a enable the server 320 to identify the device identifier of the primary client device 100, identify and execute the game instance requested by the primary client device 100, and provide streaming video frames for forwarding to the primary client device for rendering. The multiple modules of the handover application module 355 enable the server to discover and qualify one or more secondary client devices and provide a list of qualified secondary client devices as an option for device handover.

[0077] Some of the modules within the game application 321-a, which is part of the server-side application 321, are used to provide game-related data to the client device 100, including a primary client device identifier 322, a game execution engine 324, and an encoder 328, to name a few. Modules within the handover application module 355 include modules for implementing device handover and modules for profiling one or more secondary client devices. Modules for implementing device handover include a handover manager 330, a resume manager 332, and a pause manager 334, to name a few. Modules for profiling one or more secondary client devices include a client device profiler 340.

[0078] In response to the user selecting a particular game title for game play, the cloud gaming system 300 identifies a data center 310 located in or near the geographic location of the host client device and configured to service the game play request. The cloud gaming system 300 forwards the game play request to a server at the identified data center, which identifies the game title from a game title data store 314 located within the identified data center.

[0079] As part of the service request, the server 320 uses the primary client device identifier 322 to identify the client device 100 initiating the game play request and designates the client device 100 as the primary client device 100. The identification of the primary client device 100 is used to forward the streaming content. The server 320 then establishes the game play session by providing the necessary resources for game execution (e.g., storage, processor, etc.) and directing the game execution engine 324 of the game application 321-a to retrieve the game code for the game and instantiate the game by executing the game code for the game on the server 320. As part of instantiating the game, the game execution engine 324 generates video frames 326 for the game. The video frames 326 are compressed using an encoder 328 according to a communication protocol defined for the primary client device 100. The encoded game data is streamed to the primary client device 100 for rendering. In some embodiments, the encoder 328 may be part of the game application 321-a. The application programming interface (API) of the game application 321-a is used to stream the encoded video frames to the primary client device 100. In some embodiments, an API included with the game application 321-a may include encoding logic for encoding video frames and forwarding the encoded video frames to the primary client device. In alternative embodiments, the video frames generated by the game execution engine 324 are captured and processed by the operating system of the server 320. As previously described, in some cases, the operating system may encode the video frames, monitor network conditions, and adjust the quality of the encoded video frames before streaming them to the primary client device 100. Alternatively, the operating system may process the video frames and forward them as an HDMI video signal. An external application outside the cloud gaming system may capture the HDMI video signal, encode the video frames captured in the HDMI video signal using an external encoder, monitor network conditions, and adjust the resolution and / or transmission rate of the images in the video frames before transmitting them to the primary client device 100 via one or more APIs available within the external application. It should be noted that the process for capturing, monitoring network conditions, encoding, and transmitting video frames can also be extended to audio data. User interactions at the primary client device 100 are received by the server and used to adjust the game results at the game execution engine 324.

[0080] During a game play session, the primary client device 100 may actively seek out other client devices (also referred to herein as secondary client devices) that are identified as being local to the primary client device 100. In one embodiment, upon detecting a secondary client device, the primary client device 100 may establish a network connection with the secondary client device and send a signal to the secondary client device to communicate with and be profiled by the server 320. Due to the presence of security devices (such as firewalls) at the server 320, it may be advantageous to have the secondary client device establish the network connection rather than the server 320. Establishing the network connection through the secondary client device may be accomplished through one or more background agents available on the secondary client device. For example, secondary client devices are typically always connected to social media servers, email servers, and the like, and receive social media or email notifications. The primary client device can access the secondary client device via this network connection and send a signal to the secondary client device to profile itself. The signal may include a link to the server, the server's IP address, etc., to enable the secondary client device to identify and establish a communication link with the appropriate server to initiate the profile request 105. In an alternative embodiment, upon detecting a secondary client device, the primary client device 100 may send a signal with a profile request 105 to the server 320, wherein the profile request 105 requests the server to establish a communication link with the secondary client device and generate a profile for the secondary client device detected by the primary client device. In this embodiment, the signal from the primary client device 100 may include a device identifier of the secondary client device.

[0081] In response to a signal from a primary client device or an auxiliary client device, the server 320 may establish a communication connection with the auxiliary client device. This communication connection allows the client device profiler 340 of the server-side application 321 to request and receive information from the auxiliary client device, such as network bandwidth information, quality of service information, device capabilities (including memory, processor speed, etc.), device availability, game / application availability, etc. The exchanged information is used to generate a profile for the auxiliary client device. The profile identifies the device attributes of the auxiliary client device based on the information exchanged with the server. When the primary client device discovers additional auxiliary client devices, the client device profiler 340 further profiles the additional auxiliary client device in response to a profile request. The client device profiler 340 uses the profile information of the different auxiliary client devices to generate a list 342 of auxiliary client devices available for device handover.

[0082] In alternative embodiments, instead of the primary client device actively seeking out auxiliary client devices, various components of the server may actively seek out auxiliary client devices local to the primary client device that are actively participating in a game play session for a game available on the cloud gaming system. For example, a game application 321-a or an API within the game application 321-a, an API within a game server that processes video frames, or an external application within the cloud gaming system that processes video frames may actively participate in discovering auxiliary client devices for device handover. In such embodiments, the server may determine the geographic location of the primary client device using signals generated by the primary client device, or based on information obtained from social media posts with which a user interacts using the primary client device (before or during a game play session), tools embedded in the primary client device, or applications implemented within the primary client device. For example, using the geographic location of the primary client device, the server may use one or more device discovery protocols implemented in various client devices to seek out one or more auxiliary client devices identified as being local to the primary client device. In some embodiments, the auxiliary client devices are registered with the cloud gaming system and are either actively interacting with the cloud gaming system or in a dormant state. In other embodiments, the secondary client device may actively interact with other content provider systems, but may be set up for cloud gaming on the cloud gaming system.

[0083] Once the secondary client devices are identified, the client device profiler 340 can generate profiles for the primary and secondary client devices, and this profiling can be done automatically when the server detects the secondary client devices or in response to a triggering event (e.g., a low battery signal, increased latency, a calendar event, etc.). The client device profiler 340 can use the profiles for the primary and secondary client devices to determine whether any one or more of the secondary client devices have device attributes that can enhance the gaming experience for the user's currently playing game. When it is determined from the generated device profiles that one or more secondary client devices include device attributes that can provide a better gaming experience (enhanced resolution, improved frame rate, lower latency, etc.) than that provided by the primary client device, the client device profiler 340 identifies selected ones of the secondary client devices and generates a list of the secondary client devices.

[0084] The auxiliary client device list 342 is forwarded to the handover manager 330. The handover manager 330 is configured to query the game execution engine 324 to determine the game's device attribute requirements. Each game may have its own device attribute requirements. For example, a graphics-intensive game may require higher network bandwidth, while a computationally intensive game may require a faster processor. As a result, device attribute requirements may be game-specific, and auxiliary client devices may be identified based on the game's device attribute requirements. The handover manager 330 uses the game's device attribute requirements to limit the auxiliary client devices provided in the list 342. In addition, in some embodiments, the handover manager 330 may identify one or more device attributes or services that need to be preconfigured on the auxiliary client device in order for the auxiliary client device to be eligible to play the game. The handover manager 330 may then proceed to preconfigure the required device attributes or services in order to qualify the auxiliary client device to play the game. The handover manager 330 refines the list 342 to include only those auxiliary client devices that are eligible to play the game. The auxiliary client devices identified in the refined list may be a subset of the auxiliary client devices provided by the client device profiler 340.

[0085] It should be noted that the subset of eligible secondary client devices may include one secondary client device from the list 342 provided by the client device profiler 340, some of the secondary client devices from the list, or all of the secondary client devices in the list. The handover manager 330 forwards this refined list of eligible secondary client devices to the primary client device 100 for user selection with a handover option, as indicated by the directional arrow represented by bubble 'a'.

[0086] A handover option with a list of eligible auxiliary client devices is rendered on the display of the primary client device. In one embodiment, the list of eligible auxiliary client devices can be rendered as a client-side overlay-style menu on the game scene, and the primary client device can be provided with an option to select any of the eligible auxiliary client devices. In another embodiment, the list of eligible auxiliary client devices can be rendered on the server side. In such an embodiment, the list of eligible auxiliary client devices can be part of the video stream forwarded to the client device for rendering. The user selection of the handover option and / or eligible auxiliary client device is transmitted to the handover manager 330 via the network 200, as indicated by the directional arrow represented by bubble 'b'. In one embodiment, in response to receiving the user selection from the primary client device, the handover manager 330 may then proceed to instruct or send a command or signal to the pause manager 334 to pause game play, as indicated by the directional arrow represented by bubble 'c'. The pause command or instruction will include the identifier of the primary client device and the auxiliary client device identified in the user selection. In response to the pause command, the pause manager 334 signals the game execution engine 324 to pause game play. In an optional embodiment, the handover manager 330 can directly send a signal to the game execution engine 324 to suspend the game. In one embodiment, no matter which module provides the pause signal, the game execution engine 324 will interact with the state data manager 338 in response to the pause signal to determine the game state of the game when the pause request is received. The state data manager 338 is configured to analyze the game play of the game to determine the game state of the game when the pause signal is received, and identify the restart point at which the game play can be resumed. The game state and restart point of the game can be maintained by the state data manager 338 for the duration of the game play session. In an optional embodiment, the state data manager 338 can write the saved data of the game, which includes the progress made in the game during the game play until the time point when the pause request is received. When the game play is resumed, this saved data can be used to recreate the game play of the game. In addition, as part of pausing the game, the game execution engine 324 suspends streaming video frames to the main client device identified in the pause signal.

[0087] In addition to providing the pause signal, the handover manager 330 may also provide a resume option to the secondary client device identified in the pause signal to allow the secondary client device to resume game play of the game. User selection of the resume option from the secondary client device causes a resume request signal to be forwarded from the primary client device 100 to the handover manager 330 on the server.

[0088] The handover manager 330 can verify that the resume request originates from the selected auxiliary client device. After verification, the handover manager 330 can forward the resume request to the resume manager 332 for processing. The resume manager 332 can interact with the game execution engine 324 to determine the game state of the game. The game execution engine 324 receives the request from the resume manager 332 and queries the state data manager 338 to obtain the game state and the game restart point for the game. The resume manager 332 can then instruct the game execution engine 324 to resume gameplay of the game and stream video frames from the restart point identified for the game to the selected auxiliary client device. As part of resuming the game, the resume request can include the capabilities of the selected auxiliary client device, such as screen display resolution, available input controls, audio capabilities, etc. The capabilities of the selected auxiliary client device provided with the resume request can be used to adjust the video frame stream so that the video frames can be rendered on the selected auxiliary client device. Furthermore, the game play can be adjusted using input methods available to the auxiliary client device so that the game can recognize interactions provided using the input methods available on the auxiliary client device. Before being streamed to the secondary client device, the video frames are compressed according to a communication protocol defined for the secondary client device 326. The compression techniques used to compress the video frame streams sent to the primary and secondary client devices can be the same or different, depending on the type of primary and secondary client devices selected for the game.

[0089] At the same time, the handover manager 330 updates the primary client device that will interact with the game execution engine 324 using the secondary client device identifier provided in the resume request. The handover manager 330 instructs the update primary client device module 336, which in turn sends a signal to the primary client device identifier module 322 to update the client device identifier with the secondary client device identifier, as shown in bubble 'd', so that the game execution engine can identify the client device to begin streaming encoded video frames. As part of the update, the primary client device identifier module 322 replaces the primary client device identifier with the secondary client device identifier and designates the secondary client device as the primary client device 100.

[0090] In some embodiments, the handover option may allow for the selection of more than one auxiliary client device from a list of eligible auxiliary client devices for device handover. In such embodiments, a user's selection of the resume request option from a particular auxiliary client device will determine which auxiliary client device will stream video frames from the resumed game. Selecting more than one auxiliary client device may be an additional option to provide the user with different eligible devices so that the user can select a device based on device availability, network bandwidth, communication signal strength, available frame rate, quality of service, etc. For example, two auxiliary client devices may be identified as having similar device attributes, but one may be unavailable for the entire duration of the game session from the time the handover request is received, or may have low communication signal strength or power. In such cases, providing the resume request option on both auxiliary client devices can help the user switch to the appropriate auxiliary client device for playing the game during device handover.

[0091] Figure 3The diagram shows options for identifying auxiliary client devices within a primary client device in one embodiment of the present invention, as well as various modules for selecting an auxiliary client device for device handover. The primary client device is a computing device that includes components such as a processor 104 for processing requests and data received from a server (such as a cloud gaming server 320), one or more memory units for storing data, and network components and communication protocols for connecting to the Internet and interacting with other devices (such as servers, consoles, other client devices, controllers, network devices, etc.). The primary client device 100 can be a laptop computer, desktop computer, mobile phone, tablet computer device, head-mounted display, etc. The user login module 102 is used to provide a user interface for rendering on the display of the primary client device 100 and for providing user authentication information for use, for example, in accessing cloud gaming services. The user interface can also be used to provide user selection and user interaction during game play. The processor 104 includes a stream processor 106, which is used to receive and process streaming content received from a server (such as a cloud gaming server 320 or other content provider) via the network 200. The stream processor 106 includes the necessary logic for verifying that the streaming content received from the server complies with the communication protocol defined for the primary client device 100 and for processing the content. The processor 104 may use a decoder 108 available to the processor to decompress the encoded streaming data provided by the server 320. The decompressed data is forwarded to a data processing program 110 within the processor 104 for analysis to identify and separate the various data components of the content. The streaming content (i.e., streaming video frame data) from the server 320 may include video frame data, audio data, haptic data, images, etc. The data processing program 110 separates the various data components provided in the streaming content and forwards the different data components to the appropriate device components on the primary client device 100 for rendering. For example, the audio data component 110-a may be forwarded to a speaker for rendering, the video data component 110-b and the image data component may be forwarded to a display for rendering, and so on. User input (such as user selections, game interactions, etc.) provided at the client device in response to the streaming content and captured by the user input module 107 is processed by the processor 104 and forwarded to the server 320 to, for example, drive the outcome of a game.

[0092] The primary client device 100 also includes a handover module 120 that is configured to proactively seek out and detect secondary client devices located in the vicinity of the primary client device 100 in order to present a handover option for handing over the devices for gaming. The handover module 120 includes a number of submodules, such as a discovery agent 130, a device profile indicator 140, and handover logic 150, to name a few. The discovery agent 130 is configured to discover other client devices that are identified as being local to the primary client device. The discovery agent 130 uses various signals and / or protocols emitted by the client devices to seek out other client devices. Reference will be made to Figure 4 The discovery agent 130 is described in more detail.

[0093] In some embodiments, the device profile indicator 140 is used to signal the secondary client devices discovered by the discovery agent 130 to establish a communication connection with the server of the cloud gaming system and be profiled. In other embodiments, the device profile indicator 140 may signal the server of the cloud gaming system to establish a communication connection with the discovered secondary client devices and profile each of the secondary client devices. Figure 5 The role of the device profile indicator 140 is discussed in more detail.

[0094] The handover logic 150 is used to provide an appropriate user interface for rendering the handover options and the list of secondary client devices that are eligible to play the game in order to successfully perform the device handover. Figure 6A 、 Figure 6B 、 Figure 6C-1 and Figure 6C-2 The functionality of the handover logic 150 is discussed in greater detail.

[0095] Figure 4The various submodules of the discovery agent 130 that can be used on the client device 100 to seek and discover other client devices are shown. The submodules of the discovery agent 130 can be roughly divided into two main submodules: a device location detector 132 and a service discovery module 134. The device location detector 132 is configured to identify the geographic location of a primary client device by using signals emitted by the primary client device or by obtaining data from components within the primary client device or devices external to the primary client device. In a local network (such as a home network), various client devices (such as tablet computers, smart TVs, smart watches, mobile phones, cable boxes, game consoles, etc.) are connected to the home network via Ethernet connections (i.e., wired connections), Wi-Fi (i.e., wireless connections), or a combination of both. As a result, the geographic location of the primary client device can be determined. In one embodiment, this is done by evaluating the Wi-Fi signal strength presented by the primary client device to one or more network access points within the network. This evaluation can be performed using a Wi-Fi signal detector 132-a. In an alternative embodiment, a local router device detector 132-b can query a local router device to obtain a list of client devices connected to the network through the router device. In yet another embodiment, a global positioning service (GPS) tool 132-c implemented in the master client device 100 can be used to pinpoint the master client device's geographic location. In yet another embodiment, the master client device's geographic location can be determined by interrogating signals between the master client device and one or more cell towers using a cell tower signal detector 132-d. This mode of detecting the master client device's geographic location can be useful when the master client device is configured to interact with a cell tower, such as a mobile phone. In some embodiments, a combination of various device location detectors 132 can be used to determine the user's master client device 100's geographic location.

[0096] Once the geographic location of the primary client device is determined, one or more service discovery modules 134 can be used to proactively seek and discover other client devices in the same network environment as the primary client device 100. For example, a Domain Name System (DNS)-based protocol (such as the multicast DNS service discovery protocol 134-a) implemented on various devices such as tablets, laptops, printers, etc. can be used to discover secondary client devices available within the network in which the primary client device operates. Alternatively, a service location protocol 134-b can be used to detect secondary client devices within the local area network in which the primary client device operates. Other service discovery protocols may include the Bluetooth service discovery protocol 134-c, which is used to detect Bluetooth-enabled secondary client devices within the local area network of the primary client device using wireless technology. The list of service discovery protocols provided here is merely an example, and other service discovery protocols can be used to automatically seek and discover secondary client devices within the local network in which the primary client device operates. For example, various secondary client devices discovered using different protocols are forwarded to the server 320 via the network 200 using the wireless connection 138. The server may receive the device identifiers of the discovered auxiliary client devices and establish a communication connection to exchange information used to identify device attributes and profile the corresponding auxiliary client devices. Alternatively, a signal may be sent to various auxiliary client devices via a request to establish a communication connection with the server 320 and exchange information with the server. The information exchanged with the server is used to generate a profile for the auxiliary client device. The profile generated for the auxiliary client device is used to define the auxiliary client device so that the appropriate auxiliary client device can be identified and presented as an option for device handover.

[0097] Figure 5 An exemplary client device profiler 340 is shown in one embodiment for limiting secondary client devices that are found local to a primary client device in order to provide options for qualified secondary client devices for device handover. Figure 2The client device profiler 340 receives one or more signals to profile one or more auxiliary client devices discovered by the primary client device 100 using various device discovery protocols. Such signals may be received from the client device's device profile indicator 140 or from one or more auxiliary client devices. In response to the one or more signals, the client device profiler 340 establishes a communication connection and begins exchanging information with the corresponding auxiliary client device. Various submodules within the client device profiler 340 may be used to obtain information from the auxiliary client devices. For example, a Quality of Service (QOS) evaluator 341 may be used to obtain QOS information from the auxiliary client devices. QOS information may include network bandwidth information, frame rate, streaming capabilities, and other network-related information that impacts the user's gaming experience, such as ping time, packet loss, link type (i.e., network connection using WiFi, Ethernet, Long Term Evolution (LTE) technology, etc.). In some embodiments, QOS information may be used to organize the auxiliary client devices discovered by the primary client device. In other embodiments, information exchanged between the auxiliary client devices and the client device profiler 340 may be used to organize the auxiliary client devices. For example, device attributes may be ranked according to the device attribute requirements of a game. Processing-intensive games may rank processor speed higher than other device attributes, and graphics-intensive games may rank network bandwidth higher than other device attributes. The QOS evaluator 341 may rank device attributes according to the requirements of the game and use this ranking of device attributes to identify and qualify auxiliary client devices discovered by the primary client device for device handover. In some embodiments, the list of qualified auxiliary client devices is organized in a list based on how closely the device attributes of the auxiliary client devices match the ranked device attributes.

[0098] Device profile discovery 349 may be used to determine the hardware and software specifications of the secondary client device, including, but not limited to, the type and version of the operating system, the type of processor, the speed of the processor, the type and amount of memory, etc. The device profile may be used to classify the secondary client device and to package streaming video frames for forwarding to the secondary client device.

[0099] The device availability module 343 can be used to determine the availability of each of the auxiliary client devices discovered by the main client device. Device availability can drive the selection of qualified auxiliary client devices so that they are provided as options for device handover. Some or all of the auxiliary client devices can be connected to the local network when discovered. One or more of the auxiliary client devices can be entrusted to provide services for events (such as calendar events, user action events, etc.). When the event occurs, the resources of one or more auxiliary client devices entrusted to provide services for the event can be used during the event, and these auxiliary client devices are shown to be in a 'busy' state. In order to determine whether a specific auxiliary client device is available during device handover, the device availability module 343 is configured to query an event mechanism (not shown) to determine the various events to which the resources of the auxiliary client device can be used or entrusted. In one embodiment, the event mechanism can be configured to query and obtain information related to the event from various scheduling sources so that during a game session, an auxiliary client device for device handover can be identified.

[0100] It should be noted that upon receiving the handover option from the primary client device, the device availability information can be used to determine which of the secondary client devices are available during the device handover. Alternatively, the device availability information can be used to determine which of the secondary client devices are scheduled to be busy with a game play session that is scheduled for a future time. An interface (such as a network interface 347) can be used to provide interface logic and / or APIs for querying different scheduling applications (such as a calendar application 345, a social media application 346, an email application, etc.) to determine whether any event has been scheduled at the current time or is being scheduled for a time that has already committed or requires the resources of a specific secondary client device for a game play session. For example, if the user is currently committed to playing an online multiplayer game through social media posts, the event mechanism may be able to extract such information by querying the social media application 346 that relies on one or more social graphs 346-a to identify social contacts. The identities of the social contacts can be used to determine which social posts / feedback to monitor or query to obtain the user's social commitment to play games. Similarly, if a user has a calendar entry for playing an online multiplayer game every Friday at 8:00 PM, the event mechanism may be able to extract this information from the calendar application 345, or from a calendar option in an email application or social media, or other scheduling resources. The device availability module 343 analyzes the various events to which a particular auxiliary client device has been committed to determine which of the identified auxiliary client devices are available for inclusion in the device handover option. The scheduling information obtained from the device availability module 343 is used to identify and qualify the auxiliary client devices that are offered as options with the handover option. Of course, the identified auxiliary client devices have device attributes that are used to provide the user with an optimal gaming experience.

[0101] The game / application discovery module 344 is used to determine the various applications and / or games available at the data center that is assigned to the game playing session. Figure 1 As shown, each data center includes multiple games that are available or configured for play. Multiple servers provide the necessary resources for playing the games available in each data center. The game / application discovery module 344 is configured to determine the list of available games and whether any games need to be preconfigured at the data center to enable game play. If a particular game is not available at the data center, the cloud gaming system 300 can preconfigure the game at the data center so that when and if the game is selected for play, an instance of the game can be retrieved from the game name data store 314, the instance of the game can be executed on the server in the data center, and the game content can be streamed to the primary client device and the secondary client device. Similarly, if the secondary client device does not have one or more device attributes or if the device attributes do not meet the required level for game play, the game / application discovery module 344 can identify such attributes and use the cloud gaming system 300 to preconfigure these attributes to make the secondary client device eligible for game play. The client device profiler 340 uses the information collected by the various sub-modules to generate device profiles for the various secondary client devices that are discovered to be local to the primary client device.

[0102] Figure 6A An exemplary device profile generated by client device profiler 340 for all auxiliary client devices identified as local to the primary client device is shown. The device profile uses device identifiers to identify the auxiliary client devices and lists various device attributes of the auxiliary client devices. For simplicity, the device identifiers of the auxiliary client devices are listed in numerical order; in reality, a device identifier is a unique identifier that can be used to clearly identify each client device. The number of auxiliary client devices and the specific device attributes are provided as examples and should not be considered limiting. A fewer or greater number of device attributes may be identified in the device profile generated by client device profiler 340.

[0103] The configuration file generated by the client device profiler 340 can be used by the handover manager 330 at the server 320 to define secondary client devices and identify specific ones of the secondary client devices that are eligible to be provided with handover options for device handover during game play of the game. The eligible secondary client devices are returned to the primary client device 100 along with the handover options for rendering and user selection.

[0104] Figure 6B 、 Figure 6C-1 and Figure 6C-21 shows different examples of user interfaces rendered on a display of a primary client device. The handover logic 150 at the primary client device 100 receives a handover option and a list of eligible secondary client devices from the handover manager 330 at the server 320 and presents the handover option along with the list of eligible secondary client devices in a user interface 153 for user selection. The user interface identifies various options available to the user (i.e., "Pause" or "Resume"). In addition, the user interface provides a device identifier for the primary client device (D1), as shown in box 155, and a list of eligible secondary client devices under a "Select Another Device" option 157. Figure 6B The user interface 153 is shown rendered on the display of the primary client device in one embodiment. The list of secondary client devices provided under option 157 for user selection is a list of secondary client devices provided by the client device parser 340 (in Figure 6A ), and in one embodiment, are organized according to device attributes of corresponding auxiliary client devices as compared to the device attribute requirements of the game.

[0105] A user selection of the "Pause" option (as indicated by the black cursor) causes a signal to be sent to the handover manager 330 at the server 320 to pause gameplay of the game. In response, the handover manager 330 causes an instruction to be sent to the game execution engine 324 to pause streaming of video frames to the primary client device. The handover manager 330 also sends a signal to the pause manager 334 to save the game state of the gameplay so that it can be restarted from the point where gameplay was paused. The game state of the gameplay is saved by the state data manager 338. A user selection of a secondary client device (as indicated by the gray dotted cursor) causes the secondary client device to be identified as the handover device to which gameplay must be streamed if and when the user decides to resume the game on the selected secondary client device. A user selection of the Pause option also causes a "Resume" option to be forwarded to the selected secondary client device. The user selection of the Resume option at the secondary client device is forwarded to the handover manager 330 at the server 320. The handover manager 330, in turn, instructs the Resume manager 332 to resume gameplay of the game. The resume manager 332 will instruct the game execution engine to retrieve the game state from the state data manager 338 and restart the game from the restart point saved in the state data manager 338. The handover manager 330 sends a signal to update the primary client device identifier to reflect the selected secondary client device identifier so that the game execution engine 324 can stream video frames for the restarted game using the device identifier of the selected secondary client device.

[0106] Figure 6C-1 and Figure 6C-2An alternative method of selecting an auxiliary client device as a handover device in one embodiment of the present invention is provided. Figure 6C-1 As shown, instead of selecting the "Pause" option, the user may select one of the auxiliary client devices (as indicated by the black cursor) from the select device option 157 for device handover. User selection of an auxiliary client device will automatically cause the selected device to be highlighted in the select device option 157 and the pause option to be activated, as shown. Figure 6C-2 The process of pausing game play, saving the game state, and providing a "resume" option in response to a user selection on a secondary client device is similar to that described in reference to FIG. Figure 6B Game play may be resumed if and when the user selects the resume option from the secondary client device.

[0107] The various embodiments discussed herein allow a primary client device to actively seek and discover secondary client devices local to the primary client device using various protocols and signal detectors implemented in each of the client devices. In some embodiments, the protocols and technologies available within each client device enable an actively participating client (i.e., a primary client device) receiving streaming content from a content provider (e.g., a game server) to discover other dormant client devices eligible to receive streaming content, other idle client devices that can be awakened to become clients that can receive streaming content (e.g., client devices with idle sockets waiting for messages), or standby clients that are eligible to receive streaming content but are currently performing limited work (e.g., recording a program or downloading certain content). In some embodiments, the actively participating client can prompt other client devices to exchange information (e.g., session information, QOS information, device capabilities, ongoing games, video codec information, etc.) and initiate preparations for the streaming session. This can include monitoring network bandwidth. If the actively participating client is on a desktop computer with low battery, a trigger event (i.e., a low battery trigger event) may occur, prompting a user interface to appear on the tablet computer, suggesting that the user switch games or move them to another client device. The suggestions may include other client devices local to the tablet computer that may be used to play the game, and these client devices may be included in the suggestions based on their profiles.

[0108] Various methods can be used to determine the location of the master client device. For example, in one embodiment, for a wireless master client device, the Wi-Fi signal strength to one or more access points within the home network or even a neighbor's access point can be used to determine the location within the user's home. In other embodiments, GPS data can be used to determine the location of the master client device. Such location information can be used, for example, to determine whether the user is entering a room where another client device resides. The user entering the room can act as a trigger to test the network connection of the other client devices used to stream content. If the test identifies that the QOS of the other client device is indeed good, then a device handover option can be displayed on the master client device, thereby notifying the user that the other client device may be able to provide a better gaming experience (e.g., higher resolution, higher frame rate, shorter waiting time, etc.), and suggesting that the user switch. The user's selection of the other client device at the displayed option will cause the switch to occur.

[0109] Typically, various personal client devices are continuously connected to network services, such as gaming networks, social media networks, content provider networks, and the like. Such network services may keep track of a user's various personal client devices, their geographic locations, and network connections. For example, if a user is playing a game on a mobile phone using an online service (such as a gaming network), the online service may be able to use signals generated by the mobile phone (such as GPS signals, cell tower signals, Wi-Fi hotspots, etc.) to determine the location of the client device and communicate with other client devices in a private or local network (such as the user's home network). Other client devices in the local or private network (e.g., home network) may periodically connect to the gaming network or other online service and be aware of the geographic location of the user's mobile phone being used for gaming. If the gaming network detects that the user of the mobile phone is close to home, this may initiate a trigger event to verify the network bandwidth and other QoS attributes of the mobile phone, as well as the network bandwidth of other client devices in the home network. If and when the user requests a device handover, this pre-established device attributes of the mobile phone and other client devices in the network may facilitate faster and more frictionless device switching, as the preliminary work of defining the device has already been completed before the handover request is received.

[0110] In some embodiments, the event mechanism may query calendar events, for example, to identify reservations made by users for online multiplayer gaming sessions via the gaming network. Identification of calendar events may trigger preparation of various client devices (a primary client device and one or more secondary client devices) for the streaming session. Assuming the client devices are not already being used to stream content, the mere detection of a scheduled event may trigger profiling and qualification of client devices (primary and secondary) in the local network. This may include pre-configuring services and properties to prepare the primary client device and / or one or more secondary client devices for the streaming session in advance, and this pre-configuration is done in anticipation of selecting at least one of the client devices for gaming when the calendar event occurs.

[0111] Various embodiments describe ways to enable a current primary client device to opportunistically contact all potential secondary client devices to which a user can stream a game session. Specifically, the primary client device can engage in background QOS and profiling of secondary client devices to obtain the status of various device properties, and this background QOS and profiling can be done at specific times, such as when the primary client device is detected to be idle, or in a fixed duty cycle, or periodically or in response to a triggering event. This profiling is done before the user initiates a change, so that if and when the user decides to select the device handover option, a faster, frictionless change can occur because the secondary client devices have already been profiled, and in some embodiments, codec settings and / or other device properties have already been pre-established or can be pre-established for fast stream switching.

[0112] If a particular secondary client device has a usage pattern, then this usage pattern may affect the QOS. By profiling the secondary client device, the primary client device is made aware of this usage pattern, which can help qualify the secondary client device for handover. For example, a profile may identify secondary client devices with a trust interval that may be known to have a high QOS. When and if the secondary client device is selected for device handover, this information can be used to call pre-tested codecs and other stream settings (i.e., device properties that enable streaming of video frames or content). In another example, if the secondary client device has an unpredictable process load (CPU load) during working hours, then a new QOS test may have to be performed each time a handover event is initiated.

[0113] In some embodiments, profile information generated for various auxiliary client devices can be used to determine whether other auxiliary client devices exist on a local, private, or home network that are already configured to receive streamed video frames of the game currently being directed to the primary client device. A handover option can be presented to the user with the device identifier of the auxiliary client device that is already configured to receive streamed video frames of the game. In some embodiments, a new QOS test can be performed before performing the handover. In alternative embodiments, existing QOS tests used to qualify auxiliary client devices can be relied upon, and device handover can be performed without performing additional QOS tests, as long as the auxiliary client device and the primary client device remain on the same network. In some embodiments, the primary client device can be any one of a personal computer, tablet computer, mobile phone, desktop computer, television, etc. The auxiliary client device can be a head-mounted display. Alternatively, the primary client device is a head-mounted display, and the auxiliary client device selected for device handover can be any one of a personal computer, tablet computer, mobile phone, desktop computer, television, etc. In other embodiments, both the primary and auxiliary client devices can be head-mounted displays.

[0114] In one embodiment, for more information on how to use a user's in-game progress to recreate the game state of a game, reference may be made to U.S. application Ser. No. 12 / 917,388, filed on Nov. 1, 2010, entitled "USER INTERFACE, SYSTEM AND METHOD FOR CONTROLLING A VIDEO STREAM," the entire contents of which are incorporated herein by reference. The process for recreating the game state defined above is one possible approach, and other approaches to recreating the game state of a game may also be employed. For more information on video generation and distribution, reference may be made to U.S. application Ser. No. 12 / 790,955, filed on May 31, 2010, entitled "GAME EXECUTION ENVIRONMENT," (hereafter issued as U.S. Patent No. 8,506,402), the entire contents of which are incorporated herein by reference. For more information regarding server-side generation and / or rendering of game videos, reference may be made to U.S. application Ser. No. 12 / 791,819, filed on Jun. 1, 2010, and entitled “QUALIFIED VIDEO DELIVERY,” which is incorporated herein by reference in its entirety. For more information regarding server-side execution of computer programs, reference may be made to U.S. application Ser. No. 13 / 231,751, filed on Sep. 13, 2011, and entitled “ADD-ON MANAGEMENT SYSTEMS,” which is incorporated herein by reference in its entirety.

[0115] Figure 7 Various method operations are illustrated for identifying options for a secondary client device for device handover for game play in one embodiment of the present invention. The method begins at operation 710, where a game play session for a game is established for the primary client device in response to a request to play a game received from a primary client device. The primary client device can be used to access a cloud gaming system through a user account to identify a game for game play. A server in the cloud gaming system authenticates the user and the game play request and establishes the game play session by executing a game instance for streaming video frames of the game to the primary client device. The video frames may be encrypted using an encoder available on the server before being streamed to the primary client device.

[0116] A request is received to generate a configuration file for one or more secondary client devices, as shown in operation 720. The primary client device may discover the secondary client devices using signals generated by the secondary client devices and / or through one or more device discovery protocols implemented within the secondary client devices. The primary client device may initiate a request to generate a configuration file by sending a signal to the secondary client device to establish a communication connection with the server and be parsed. Alternatively, the primary client device may send a signal to the server with a device identifier of a secondary client device discovered to be local to the primary client device, requesting the server to establish a communication connection with the secondary client device and exchange information to generate a configuration file for the corresponding secondary client device. In response to the signal, the server uses the established communication connection to exchange information with the secondary client device, identifies one or more device attributes, and generates a configuration file for each of the secondary client devices using the corresponding device attributes. The information provided in the configuration file is used to qualify the secondary client devices for use in game play.

[0117] As shown in operation 730, a handover option is provided to the primary client during game play. The handover option is configured to identify one or more secondary client devices that are eligible for game play based on the generated configuration file. The handover option is displayed on a user interface of the primary client device for user selection.

[0118] As shown in operation 740, a user selection of a secondary client device is received from the primary client device. In response to receiving the user selection of the secondary client device, game play is paused and the game state of the game play is saved by the server. Pausing game play results in pausing the streaming of video frames to the primary client device. In response to pausing the game play, a resume option is provided on the secondary client device to resume game play on the secondary client device.

[0119] As shown in operation 750, a user selection of a resume request is received by the server from the secondary client device. In response to the resume request, the server is configured to access the saved game state of the game and continue the game play on the secondary client device. The resume request causes the server to stream video frames of the resumed game to the secondary client device. The secondary client device is designated as the primary client device so that the streamed video frames can be directed to the secondary client device instead of the primary client device. The device handover is performed quickly and seamlessly with minimal latency, thereby enabling users of the primary and secondary client devices to enjoy a comparable or enhanced gaming experience.

[0120] In some embodiments, depending on the device attributes of the selected secondary client device, device switching may allow the user to have a better gaming experience than that provided by the primary client device, such as higher resolution, higher frame rate, shorter latency, etc. Alternatively, device switching may provide a way to continue game play without a significant change in the gaming experience. This may be the case, for example, when device switching is implemented in response to a triggering event, such as a low battery detected in the primary client device.

[0121] Although the method operations are described in a particular order, it should be understood that other housekeeping operations may be performed between the operations, or that the operations may be adjusted so that they occur at slightly different times, or may be distributed in a system that allows processing operations to occur at various time intervals associated with the processing, so long as the processing of the superimposed operations is performed in the desired manner.

[0122] Figure 8An embodiment of an information service provider architecture that can be used to provide access to different games is shown. An information service provider (ISP) 1070 delivers a large number of information services to geographically dispersed users 1082 connected via a network 1086. Although various embodiments have been discussed with reference to providing quick access to games, the embodiments can be expanded to provide one or more other types of services. For example, an ISP can deliver only one type of service (such as games) or a variety of services (such as games, stock price updates, broadcast media, news, sports, competitions, etc.). In addition, the services provided by each ISP can be dynamic, that is, services can be added or removed at any time. Therefore, the ISP that provides a particular type of service to a particular individual can change over time. For example, when a user is in her hometown, the user may be served by an ISP close to the user, and when the user travels to a different city, the user may be served by a different ISP. The hometown ISP will transfer the required information and data from the user's game or access profile to the new ISP via a connection module, so that the user information "follows" the user to the new city, making the data closer to the user and easier to access. In another embodiment, a master-server relationship may be established between a master ISP, which manages the user's information, and a server ISP, which interacts directly with the user under the control of the master ISP. In another embodiment, data is transferred from one ISP to another as the client moves around the world (i.e., during a switch in the data center assigned to the user), and this transfer may be based on the compatibility of the services provided by the respective ISPs, so that the services are provided to the user in a better location than the ISP that delivers those services.

[0123] ISP 1070 includes an application service provider (ASP) 1072, which provides computer-based services to customers over a network. Software provided using the ASP model is sometimes also referred to as on-demand software or software as a service (SaaS). A simple form of providing access to a specific application (such as customer relationship management) is to use a standard protocol (such as HTTP). For example, the application software resides on the vendor's system and is accessed by the user through a web browser using HTML, dedicated client software provided by the vendor, or other remote interfaces (such as thin clients).

[0124] Services delivered over a wide geographic area often use cloud computing. Cloud computing is a computing method in which dynamically scalable and often virtualized resources are provided as a service over the Internet. Users do not need to be experts in the technical infrastructure that supports their "cloud." Cloud computing can be divided into different services, such as Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS). Cloud computing services often provide online shared business applications accessed from a web browser, while the software and data are stored on servers. The term cloud is used as a metaphor for the Internet (e.g., using servers, storage, and logic), based on how the Internet is depicted in computer network diagrams and the abstract concept of the complex infrastructure it hides.

[0125] In addition, ISP 1070 includes a Game Processing Server (GaPS) 1074, which is used by game clients to play single-player and multi-player video games. Most video games played over the Internet are operated by connecting to a game server. Typically, games use a dedicated server application that collects data from players and distributes it to other players. This is more efficient and more effective than a peer-to-peer arrangement, but it requires a separate server to host the server application. In another embodiment, the GaPS establishes communication between players, and their respective gaming devices exchange information independently of a centralized GaPS.

[0126] Dedicated GAPs are servers that run independently of the client. Such servers typically run on dedicated hardware located in a data center, providing greater bandwidth and dedicated processing power. Dedicated servers are the preferred method for hosting game servers for most PC-based multiplayer games. Massively multiplayer online games run on dedicated servers, typically hosted by the software company that owns the game title, allowing the dedicated server to control and update the content.

[0127] Broadcast Processing Server (BPS) 1076 distributes audio or video signals to viewers. Broadcasting to a very small audience is sometimes called narrowcasting. The final leg of broadcast distribution is how the signal reaches the listener or viewer, and like a radio or television station, the signal can reach the antenna and receiver from the air, or it can arrive via cable television or cable broadcast (or "wireless cable") via the station or directly from the network. The Internet can also bring radio or television to the receiver (especially using multicast), thereby allowing the sharing of signals and bandwidth. In the past, broadcasting was limited to geographical areas, such as national broadcasting or regional broadcasting. However, with the rapid spread of the Internet, broadcasting is no longer limited by geographical location, because content can reach almost any country in the world.

[0128] Storage Service Providers (SSPs) 1078 provide computer storage space and related management services. SSPs also provide regular backup and archiving. By offering storage as a service, users can order more storage space as needed. Another major advantage is that SSPs include backup services, and users will not lose all their data if their computer's hard drive fails. In addition, multiple SSPs can have copies of all or part of a user's data, allowing users to access data in an efficient manner independent of their location or the device they are using to access the data. For example, when a user is on the move, they can access personal files on both their home computer and their mobile phone.

[0129] Communications providers 1080 provide connectivity to users. One type of communications provider is an Internet Service Provider (ISP), which provides access to the Internet. ISPs connect their customers using data transmission technologies suitable for delivering Internet Protocol datagrams, such as dial-up, DSL, cable modems, fiber optics, wireless, or dedicated high-speed interconnects. Communications providers may also offer messaging services, such as email, instant messaging, and SMS texting. Another type of communications provider is a Network Service Provider (NSP), which sells bandwidth or network access by providing direct backbone access to the Internet. Network service providers may consist of telecommunications companies, data carriers, wireless communication providers, Internet service providers, cable television operators offering high-speed Internet access, and the like.

[0130] Data exchange 1088 interconnects several modules within ISP 1070 and connects these modules to users 1082 via network 1086. Data exchange 1088 can cover a small area where all modules of ISP 1070 are nearby, or it can cover a large geographic area when different modules are geographically dispersed. For example, data exchange 1088 can include Fast Gigabit Ethernet (or faster) within a cabinet in a data center, or an intercontinental Virtual Local Area Network (VLAN).

[0131] User 1082 accesses the remote service using a client device 1084, which includes at least a CPU, memory, display, and I / O. The client device can be a PC, mobile phone, netbook, tablet computer, gaming system, PDA, etc. In one embodiment, ISP 1070 identifies the type of device used by the client and adjusts the communication method used. In other cases, the client device uses a standard communication method (such as HTML) to access ISP 1070.

[0132] Figure 9is a block diagram of a gaming system 1400 according to various embodiments of the present invention. The gaming system 1400 is configured to provide a video stream to one or more clients 1410 via a network 1415. The network is connected to Figure 1 The network 200 shown is similar. Gaming system 1400 generally includes a video server system 1420 and an optional game server 1425. The video server system 1420 is configured to provide video streaming to one or more clients 1410 with the lowest quality of service. For example, the video server system 1420 can receive game commands that change the state of a video game or the visual angle within the video game, and provide the updated video streaming that immediately reflects this change to the client 1410 with minimal lag time. The video server system 1420 can be configured to provide video streaming with a variety of optional video formats (including formats that have not yet been defined). In addition, the video streaming can include video frames configured to be presented to the user at a variety of frame rates. Typical frame rates are 30 frames per second, 60 frames per second, and 1420 frames per second. However, optional embodiments of the present invention include higher or lower frame rates.

[0133] The clients 1410, individually referred to herein as 1410A, 1410B, etc., may include head mounted displays, terminals, personal computers, game consoles, tablet computers, phones, set-top boxes, kiosks, wireless devices, digital tablets, standalone devices, handheld game playing devices, and / or the like. The clients described are similar to Figure 1Clients 100-1 to 100-n. Typically, client 1410 is configured to receive an encoded video stream, decode the video stream, and present the resulting video to a user (e.g., a player of a game). The process of receiving an encoded video stream and / or decoding the video stream typically includes storing individual video frames in a receiving buffer of the client. The video stream can be presented to the user on a display integrated with client 1410 or on a separate device such as a monitor or television. Client 1410 is optionally configured to support more than one game player. For example, a game console can be configured to support two, three, four, or more simultaneous players. Each of these players can receive a separate video stream, or a single video stream can include a region of frames generated specifically for each player (e.g., based on each player's perspective). Clients 1410 are optionally geographically dispersed. The number of clients included in gaming system 1400 can vary widely from one or two to several thousand, tens of thousands, or more. As used herein, the term "game player" is used to refer to a person who plays a game, and the term "game playing device" is used to refer to a device used to play a game. In some embodiments, a game playing device may refer to multiple computing devices that collaborate to deliver a gaming experience to a user. For example, a game console and an HMD may collaborate with the video server system 1420 to deliver a game viewed through the HMD. In one embodiment, the game console receives a video stream from the video server system 1420, and the game console forwards the video stream or updates to the video stream to the HMD for rendering.

[0134] Client 1410 is configured to communicate with the server via network 1415 (similar to Figure 1 The video stream is received over a network 200 (e.g., a video stream received over a network 1415). The network 1415 can be any type of communication network, including a telephone network, the Internet, a wireless network, a power line network, a local area network, a wide area network, a private network, and / or the like. In a typical embodiment, the video stream is transmitted via a standard protocol such as TCP / IP or UDP / IP. Alternatively, the video stream is transmitted via a proprietary standard.

[0135] A typical example of a client 1410 is a personal computer that includes a processor, non-volatile memory, a display, decoding logic, network communication capabilities, and input devices. The decoding logic may include hardware, firmware, and / or software stored on a computer-readable medium. Systems for decoding (and encoding) video streams are known in the art and vary depending on the specific encoding scheme used.

[0136] Client 1410 may (but need not) also include a system configured to modify the received video. For example, the client may be configured to perform further rendering, overlay one video image on another, crop the video image, and / or similar operations. For example, client 1410 may be configured to receive various types of video frames (such as I-frames, P-frames, and B-frames) and to process these frames into images for display to the user. In some embodiments, components of client 1410 are configured to perform further rendering, shading, conversion to 3-D, or similar operations on the video stream. Components of client 1410 are optionally configured to receive more than one audio or video stream. Input devices of client 1410 may include, for example, a single-handed game controller, a two-handed game controller, a gesture recognition system, a gaze recognition system, a voice recognition system, a keyboard, a joystick, a pointing device, a force feedback device, a motion and / or position sensing device, a mouse, a touch screen, a neural interface, a camera, input devices yet to be developed, and / or similar devices.

[0137] The video stream (and optional audio stream) received by client 1410 is generated and provided by video server system 1420. As further described elsewhere herein, this video stream includes video frames (and the audio stream includes audio frames). Video frames (e.g., they include pixel information in an appropriate data structure) are configured to meaningfully constitute an image displayed to a user. As used herein, the term "video frame" is used to refer to a frame that primarily includes information configured to constitute (e.g., implement) an image displayed to a user. Most of the teachings herein regarding "video frames" can also apply to "audio frames."

[0138] Client 1410 is generally configured to receive input from the user. These inputs may include game commands that are configured to change the state of the video game or otherwise affect the game. An input device may be used to receive game commands, and / or game commands may be automatically generated by a computing instruction executed on client 1410. The received game commands are communicated from client 1410 to video server system 1420 and / or game server 1425 via network 1415. For example, in some embodiments, game commands are communicated to game server 1425 via video server system 1420. In some embodiments, a separate copy of the game commands is communicated from client 1410 to game server 1425 and video server system 1420. The communication of game commands optionally depends on the identification of the command. Game commands are optionally communicated from client 1410A via different routes or communication channels used to provide audio or video streams to client 1410A.

[0139] Game server 1425 is optionally operated by an entity distinct from video server system 1420. For example, game server 1425 may be operated by a multiplayer game publisher. In this instance, video server system 1420 is optionally viewed as a client by game server 1425 and is optionally configured to behave (from the perspective of game server 1425) as a prior art client executing a prior art game engine. Communication between video server system 1420 and game server 1425 optionally occurs via network 1415. Thus, game server 1425 may be a prior art multiplayer game server that sends game state information to multiple clients, one of which is video server system 1420. Video server system 1420 may be configured to communicate with multiple instances of game server 1425 simultaneously. For example, video server system 1420 may be configured to provide multiple different video games to different users. Each of these different video games may be supported by a different game server 1425 and / or published by a different entity. In some embodiments, several geographically distributed instances of the video server system 1420 are configured to provide game videos to multiple different users. Each of these instances of the video server system 1420 can communicate with the same instance of the game server 1425. Communication between the video server system 1420 and one or more game servers 1425 optionally occurs over a dedicated communication channel. For example, the video server system 1420 can be connected to the game server 1425 via a high-bandwidth channel that is dedicated to communication between the two systems.

[0140] The video server system 1420 includes at least a video source 1430, an I / O device 1445, a processor 1450, and a storage device 1455 (including non-transitory analog and / or digital storage devices). The video server system 1420 may include a single computing device or may be distributed across multiple computing devices. These computing devices are optionally connected via a communication system such as a local area network.

[0141] Video source 1430 is configured to provide a video stream, e.g., streaming video or a series of video frames forming a motion picture. In some embodiments, video source 1430 includes a video game engine and rendering logic. The video game engine is configured to receive game commands from the player and maintain a copy of the state of the video game based on the commands received. This game state includes the positioning of objects in the game environment and typical viewing angles. The game state may also include the properties, images, colors, and / or textures of the objects.

[0142] Game state is maintained based on game rules and game commands (such as, move, turn, attack, set focus, interact, use and / or similar commands) conventionally.The part of game engine is optionally arranged in game server 1425.Game server 1425 can maintain the copy of game state based on the game commands received from a plurality of players using geographically dispersed clients.In these cases, game server 1425 offers game state to video source 1430, wherein stores the copy of game state and performs rendering.Game server 1425 can directly receive game commands from client 1410 through network 1415, and / or can receive game commands through video server system 1420.

[0143] The video source 1430 typically includes rendering logic, such as hardware, firmware, and / or software stored on a computer-readable medium, such as storage 1455. This rendering logic is configured to create video frames of a video stream based on the game state. All or part of the rendering logic is optionally located within a graphics processing unit (GPU). The rendering logic typically includes processing stages configured to determine three-dimensional spatial relationships between objects and / or to apply appropriate textures, etc., based on the game state and viewing angle. The rendering logic generates raw video, which is then often encoded and then communicated to the client 1410. For example, the raw video may be encoded according to Adobe The raw video is encoded using a standard such as H.264, .wav, H.264, H.263, On2, VP6, VC-1, WMA, Huffyuv, Lagarith, MPG-x., Xvid., FFmpeg, x264, VP6-8, realvideo, mp3, or a similar standard. The encoding process produces a video stream, which is optionally packaged for delivery to a decoder on a remote device. A video stream is characterized by a frame size and a frame rate. Typical frame sizes include 800x600, 1280x720 (e.g., 720p), and 1024x768, but any other frame size can be used. The frame rate is the number of video frames per second. A video stream can include different types of video frames. For example, the H.264 standard includes "P" frames and "I" frames. An I frame includes information for refreshing all macroblocks / pixels on a display device, while a P frame includes information for refreshing a subset of the macroblocks / pixels. The data size of a P frame is typically smaller than an I frame. As used herein, the term "frame size" is intended to refer to the number of pixels within a frame. The term "frame data size" is used to refer to the number of bytes required to store the frame.

[0144] In an alternative embodiment, video source 1430 includes a video recording device such as a video camera. This camera can be used to generate delayed video or live video that can be included in the video stream of the computer game. The resulting video stream optionally includes both rendered images and images recorded using a still camera or a video camera. Video source 1430 may also include a storage device configured to store previously recorded video to be included in the video stream. Video source 1430 may also include: a motion or position sensing device configured to detect the motion or position of an object (e.g., a person); and logic configured to determine the game state or generate video based on the detected motion and / or position.

[0145] Video source 1430 is optionally configured to provide overlays configured to be placed on other videos. For example, these overlays may include a command interface, login instructions, messages to the game player, images of other game players, video feedback of other game players (e.g., webcam video). In embodiments where client 1410A includes a touch screen interface or a gaze detection interface, the overlay may include a virtual keyboard, joystick, touchpad, and / or similar device. In one example of an overlay, the player's voice is overlaid on an audio stream. Video source 1430 optionally also includes one or more audio sources.

[0146] In embodiments where Video Server System 1420 is configured to maintain game state based on input from more than one player, each player may have a different viewpoint, including the position and direction of the viewpoint. Video Source 1430 is optionally configured to provide a separate video stream to each player based on the player's viewpoint. Additionally, Video Source 1430 may be configured to provide a different frame size, frame data size, and / or encoding to each of Clients 1410. Video Source 1430 is optionally configured to provide 3-D video.

[0147] I / O devices 1445 are configured for use with video server system 1420 to send and / or receive information, such as video, commands, requests for information, game state, gaze information, device motion, device location, user motion, client identity, player identity, game commands, security information, audio, and / or the like. I / O devices 1445 typically include communication hardware such as a network card or modem. I / O devices 1445 are configured to communicate with game server 1425, network 1415, and / or client 1410.

[0148] Processor 1450 is configured to execute logic (e.g., software) included within the various components of Video Server System 1420 as discussed herein. For example, processor 1450 can be programmed with software instructions to perform the functions of Video Source 1430, Game Server 1425, and / or Client Qualifier 1460. Video Server System 1420 optionally includes more than one instance of processor 1450. Processor 1450 can also be programmed with software instructions to execute commands received by Video Server System 1420 or coordinate the operation of the various elements of Game System 1400 as discussed herein. Processor 1450 can include one or more hardware devices. Processor 1450 is an electronic processor.

[0149] Storage device 1455 comprises non-transitory analog and / or digital storage devices. For example, storage device 1455 may comprise an analog storage device configured to store video frames. Storage device 1455 may comprise a computer-readable digital storage device, such as a hard drive, an optical drive, or a solid-state storage device. Storage device 1455 is configured to store video frames, artificial frames, video streams including both video frames and artificial frames, audio frames, audio streams, and / or the like (e.g., in the form of an appropriate data structure or file system). Storage device 1455 is optionally distributed among multiple devices. In some embodiments, storage device 1455 is configured to store software components of video source 1430 discussed elsewhere herein. These components can be stored in a preconfigured format whenever needed.

[0150] The video server system 1420 optionally also includes a client qualifier 1460. The client qualifier 1460 is configured to remotely determine the capabilities of a client, such as client 1410A or 1410B. These capabilities may include the capabilities of the client 1410A itself and the capabilities of one or more communication channels between the client 1410A and the video server system 1420. For example, the client qualifier 1460 may be configured to test a communication channel through the network 1415.

[0151] The client qualifier 1460 can manually or automatically determine (e.g., discover) the capabilities of the client 1410A. Manual determination includes communicating with the user of the client 1410A and requesting the user to provide the capabilities. For example, in some embodiments, the client qualifier 1460 is configured to display images, text, and / or the like within the browser of the client 1410A. In one embodiment, the client 1410A is an HMD that includes a browser. In another embodiment, the client 1410A is a game console with a browser that can be displayed on the HMD. The displayed object requests the user to enter information about the client 1410A, such as the operating system, processor, video decoder type, network connection type, display resolution, etc. The information entered by the user is communicated back to the client qualifier 1460.

[0152] Automatic determination can be made, for example, by executing an agent program on Client 1410A and / or by sending a test video to Client 1410A. The agent program may include computing instructions, such as a Java script, embedded in a web page or installed as an add-on. The agent program is optionally provided by Client Qualifier 1460. In various embodiments, the agent program may discover: the processing power of Client 1410A, the decoding and display capabilities of Client 1410A, the latency reliability and bandwidth of the communication channel between Client 1410A and Video Server System 1420, the display type of Client 1410A, the presence of a firewall on Client 1410A, the hardware of Client 1410A, the software executing on Client 1410A, registry entries within Client 1410A, and / or the like.

[0153] Client Qualifier 1460 comprises hardware, firmware, and / or software stored on a computer-readable medium. Client Qualifier 1460 is optionally located on a computing device separate from one or more other elements of Video Server System 1420. For example, in some embodiments, Client Qualifier 1460 is configured to determine characteristics of a communication channel between Client 1410 and more than one instance of Video Server System 1420. In these embodiments, the information discovered by Client Qualifier can be used to determine which instance of Video Server System 1420 is best suited to deliver streaming video to one of Clients 1410.

[0154] In view of the above embodiments, it should be understood that the present invention may employ various computer-implemented operations involving data stored in a computer system. These operations include operations requiring physical manipulation of physical quantities. Any operation described herein that forms a part of the present invention is a useful machine operation. The present invention also relates to equipment or devices for performing these operations. The device may be specially constructed for the desired purpose, or the device may be a general-purpose computer selectively activated or configured by a computer program stored in the computer. In particular, various general-purpose machines may be used in conjunction with computer programs written according to the teachings of this article, or more specialized devices may be more conveniently constructed to perform the desired operations.

[0155] The invention described above may be practiced with other computer system configurations, including hand-held devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, etc. The invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network.

[0156] The present invention may also be embodied as computer-readable programming code on a computer-readable medium. Alternatively, the computer-readable programming code may be downloaded from a server using the data exchange interconnect described above. A computer-readable medium is any data storage device (including electromagnetic wave carriers) that can store data that can subsequently be read by a computer system. Examples of computer-readable media include hard drives, network attached storage (NAS), read-only memories, random access memories, CD-ROMs, CD-Rs, CD-RWs, magnetic tapes, and other optical and non-optical data storage devices. The computer-readable medium may also be distributed across a network of connected computer systems so that the computer-readable code is stored and executed in a distributed manner.

[0157] Although the foregoing invention has been described in some detail for purposes of clarity of understanding, it should be understood that certain changes and modifications may be practiced within the scope of the appended claims. The present embodiments are therefore to be considered as illustrative and not restrictive, and the invention is not to be limited to the details given herein, but may be modified within the scope and equivalents of the appended claims.

Claims

1. A method performed by a server of a cloud gaming system, comprising: In response to selecting a game for a game launched at a master client, establishing a game session for the game for the master client device, video frames of the game executed on the server being formatted according to device properties of the master client device and streamed to the master client device; generating a configuration file for a local auxiliary client device of the primary client device, wherein the configuration file identifies device attributes of the auxiliary client device; evaluating device attributes of the secondary client device to determine whether the secondary client device is eligible to play the game; providing a service to the secondary client device when the secondary client device is not eligible to play the game, wherein the provided service is selected to make the secondary client device eligible to play the game; providing a user interface having a handover option at the primary client device during play of the game, the user interface identifying the secondary client device, wherein the handover option allows selection of the secondary client device at the user interface to play the game; as well as In response to receiving a selection of the secondary client device at the user interface, switching the stream of video frames of the game from the primary client device to the secondary client device, the selection being received from the primary client device.

2. The method according to claim 1, wherein The profile for the secondary client device is dynamically generated in response to a triggering event detected at the primary client device during the gaming session.

3. The method according to claim 2, wherein: The triggering event can be initiated by the primary client device in response to detecting, at the primary client device, insufficient resources for playing the game.

4. The method according to claim 1, wherein The profile of the secondary client device is dynamically generated in response to a signal initiated by the secondary client device in response to detecting the primary client device in proximity to the secondary client device.

5. The method according to claim 1, wherein The secondary client device continues the gaming session of the game, with the secondary client device becoming the primary client device after the switch.

6. The method according to claim 1, further comprising: One or more interactive input devices for providing input to drive the game are handed over by automatically disconnecting the one or more input devices from the primary client device and automatically connecting the one or more input devices to the secondary client device.

7. The method according to claim 6, wherein: The one or more input devices are wireless input devices.

8. The method according to claim 6, wherein: One of the input devices is a wireless controller.

9. The method according to claim 1, wherein: The secondary client device is identified based on location information associated with the primary client device and the secondary client device.

10. The method of claim 1, wherein evaluating the device attributes comprises comparing the device attributes of the secondary client device to resource requirements needed to play the game.

11. The method according to claim 1, wherein Switching the stream of video frames includes formatting the video frames streamed to the secondary client device according to device properties identified in a configuration file for the secondary client device, input provided at the secondary client device during the game being interpreted to affect an outcome of the game.

12. The method of claim 11 , wherein formatting the video frames for streaming to the secondary client device comprises: Set the frame rate, or the frame size, or the frame data size, or the resolution of the image in the video frame, or the bit rate, or any combination of two or more of the above.

13. The method according to claim 1, wherein Switching the stream of video frames includes pausing the game at the primary client device and starting the game at the particular secondary client device selected for gaming.

14. A cloud gaming system, comprising: A server configured to execute a plurality of games, the server comprising: a game execution engine configured to identify game code for a game selected for play at a first client device associated with a user account, and to execute the game code to generate a stream of video frames for the game; The client device profiling module is configured to: establishing a communication connection with the first client device and a second client device in the vicinity of the first client device, querying device profiles of the first client device and the second client device to identify device attributes of the first client device and the second client device, the device attributes being used to format video frames streamed to the first client device; evaluating device attributes of the second client device to determine whether the second client device is eligible to play the game; when the second client device is not eligible to play the game, providing a service to the second client device, wherein the provided service is selected to make the second client device eligible to play the game; as well as The handover manager is configured to: A stream of video frames is switched from the first client device to the second client device, wherein device properties of the second client device are used to format the game video frames for streaming the game.

15. The cloud gaming system according to claim 14, wherein: The client device profiling module evaluates device attributes of the second client device in response to a signal received at the cloud gaming system, the signal originating from the first client device or the second client device.

16. The cloud gaming system according to claim 14, wherein: The handover manager is configured to: transmitting a pause signal to a pause manager to stop streaming the video frames to the first client device; as well as A resume signal is transmitted to a resume manager to begin streaming video frames of the game to the second client device.

17. The cloud gaming system according to claim 16, wherein: The handover manager is configured to transmit the pause signal and the resume signal in response to a trigger event detected at the first client device, or in response to a signal from the second client device.

18. The cloud gaming system according to claim 16, wherein: The game execution engine is configured to communicate with the state data manager to receive, at the second client device, a point from which the game is to be resumed based on a game state of the game.

19. The cloud gaming system according to claim 14, wherein: The handoff manager is configured to update a device identifier at the game execution engine from a first client device identifier to a second client device identifier in response to a switch in streaming of the video frames.

Images