Crowdsourced cloud games

Abstract

A system and method for executing a crowdsourced cloud application are described. The application system receives a first request from a client device to start an application session. The application system identifies a host device that satisfies the first request. The application system then starts the execution of the application session on the host device and generates several controls for the client device to control the application session running on the host device. The host device is incentivized for each hosted application session.

Description

Background Art

[0001] (Description of Related Art) Conventional cloud applications such as cloud game implementation require a significant upfront investment in server infrastructure to ensure reproducible latency and good availability of server instances. The deployment and maintenance of such infrastructure is difficult and costly, especially when the user base location is not known in advance and may change over time. These costs occur even when the user base is low, making underutilization even more costly.

[0002] Cloud service providers face significant challenges in maintaining expensive equipment within large-scale data centers. Regular upgrades are required to keep these facilities running smoothly, which can require a significant amount of time and cost. To make expenses more manageable, companies require a significant number of users to offset costs. However, directly transferring costs to users can be unreasonably expensive and unsustainable for service providers.

[0003] In view of the above, there is a need for an improved system and method for providing cloud-based services.

[0004] The advantages of the methods and mechanisms described herein can be better understood by referring to the following description in conjunction with the accompanying drawings.

Brief Description of the Drawings

[0005] [Figure 1] It is a block diagram showing one network implementation of an application system. [Figure 2] It is a block diagram showing one embodiment of various components of an application system. [Figure 3]This block diagram shows one embodiment of session restoration for a saved application session. [Figure 4] This block diagram shows one embodiment of process isolation for hosting application sessions. [Figure 4A] This figure shows a specific embodiment of process isolation for hosting application sessions. [Figure 4B] This figure shows a specific embodiment of process isolation for hosting application sessions. [Figure 4C] This figure shows a specific embodiment of process isolation for hosting application sessions. [Figure 5] This diagram illustrates an exemplary method for hosting application sessions on a host device using a peer-to-peer network. [Figure 6] This figure illustrates an exemplary method for restoring a saved application session for a client device using a host device. [Figure 7] This is an exemplary block diagram showing an application system for running cloud-based applications and a network implementation across one or more data centers. [Modes for carrying out the invention]

[0006] The following description includes numerous specific details to provide a full understanding of the methods and mechanisms presented herein. However, those skilled in the art should recognize that various embodiments can be implemented without these specific details. In some cases, well-known structures, components, signals, computer program instructions, and techniques are not shown in detail to avoid obscuring the methods described herein. For simplicity and clarity, please understand that the elements shown in the figures are not necessarily drawn to scale. For example, the dimensions of some elements may be exaggerated relative to others.

[0007] Systems, apparatus, and methods for crowdsourced cloud games are disclosed. Crowdsourced cloud applications, such as those that provide and support games, leverage end-user consumer game resources to host remote application sessions. As disclosed, a collection of end-user machines replaces the infrastructure (servers and data centers, etc.) traditionally used for cloud implementations. Furthermore, an application system is described that enables the setup and teardown of application environments on host devices. Prospective host devices are provided with the option to use the computing resources of the application system. In various embodiments, the application system deploys application files and / or other resources used by the host for running application sessions. Once a suitable host device is identified, a peer-to-peer remote streaming connection is established between the host device and the client device requesting the execution of the application session. In one embodiment, the client device can control the application session running on the host device. Furthermore, when the application session ends, files and data associated with the application session are removed from the host device. In some cases, data associated with saved applications can be stored (e.g., in a database) and restored later. For example, when a client device requests the restoration of the last saved application session, the application system can restore the application session using data stored on another suitable host device.

[0008] In one embodiment, the “application session” as described herein refers to a period of time during which a user interacts with a software application or program. During such a session, the user may engage with the application to perform various tasks or actions, for example, through a user interface. According to this implementation, an application session can begin when the user starts the application or opens a particular instance of the application on the user's device. During an application session, the user can perform various actions, such as entering data, navigating different screens or sections, executing commands, changing settings, and accessing various features or functions provided by the application. The interaction between the user and the application may also generate session-related data, such as user preferences, temporary data, or the state of the application. In one embodiment, the duration of a session can vary depending on the nature of the application and the user's purpose. For example, a session in a productivity application or a game may last for several hours, while other sessions may last only a few minutes.

[0009] In one embodiment, an application session may include gameplay. To participate in gameplay, a player connects to a game server via a network connection using either their personal computer, game console, or mobile device. For example, a user may join a virtual environment or game world where they can interact with other players who are also connected to the same server. A player's gameplay session begins when they log into the game and ends when they log out or disconnect from the server. In the following description, the terms “application session” and “session” are used interchangeably.

[0010] Figure 1 is a block diagram illustrating an exemplary network implementation of an application system. As shown in the diagram, a computing system 102 (alternatively referred to as the application system 102) is connected to a plurality of client devices 104A-104N (hereinafter also referred to as user devices 104A-104N) via a network 106. In one embodiment, the application system 102 is configured to identify an appropriate host device from a plurality of host devices 108A-108N in response to a request from a predetermined user device 104 for the execution of an application session. For example, the application system 102 may receive a request from a user device 104 to access a cloud game application. Upon receiving the request, the application system 102 identifies a predetermined host device 108 and is configured so that the requested session of the cloud game application can be hosted on the identified host device 108. Furthermore, a peer-to-peer (P2P) connection is established between the user device 104 and the host device 108, enabling the client device to remotely participate in the application session. For example, as shown in the figure, a P2P connection 130 is established between the host device 108A and the user device 104A.

[0011] In one embodiment, the host device 108 is identified as a suitable device for fulfilling a request from the user device 104 based on one or more factors. For example, the application system 102 is configured to measure end-to-end network performance metrics associated with the host device 108, such as available bandwidth and latency, to determine whether the host device 108 can execute the application session requested by the user device 104. In another example, the application system 102 is configured to determine the NAT configuration of the user device 108, including but not limited to the Network Address Translation (NAT) type (e.g., open, moderate, or restricted), nesting information, and intermediate relay requirements (e.g., STUN or TURN). In yet another example, the application system 102 may rank the host device 108 based on social credit or a rating system. For example, each user device 108 may be incentivized for successfully executing an application session and penalized for early disconnection and session interruption.

[0012] In one embodiment, the application system 102 is configured to identify whether the software associated with the requested application is available on the user device 108. Furthermore, hardware specifications, such as game performance in terms of frames per second (FPS), may also be taken into consideration when selecting a host device 108 to host the application session. In another embodiment, the application system 102 can receive a predetermined hosting schedule from each user device 108 and select a predetermined user device 108 based at least in part on that hosting schedule which is consistent with the requirements of the user device 104 to host the application session.

[0013] In one embodiment, a predetermined user device 104 is any device configured to communicate wirelessly and / or via a wired connection with an application system 102 over a network such as a network 106. In one example, a plurality of user devices 104A to 104N include one or more mobile devices, personal computers, laptops, game consoles, etc. In another embodiment, the user device 104 is configured to request the application system 102 to run the desired application if the user device 104 is unable to host the application locally due to a lack of necessary infrastructure and / or computing resources.

[0014] For example, user device 104 may send a request to application system 102 to connect to a cloud gaming application in order to access a desired game title that user device 104 cannot run due to a lack of adequate computing resources such as, for example, random access memory (RAM), graphics card, processor, operating system, and storage. In response to the request, application system 102 identifies the user associated with user device 104 by accessing user account information stored in a user data store, for example, a user database 110. Application system 102 verifies the identified user and determines one or more game titles that user device 104 is permitted to access. In one embodiment, application system 102 interacts with application database 112 to determine one or more game titles that user device 104 is permitted to access. If it is determined that user device 104 is permitted to access a game title, application system 102 identifies a host device 108 that remotely hosts a session of the game title. Furthermore, once such a host device 108 is identified, a network connection is established between the user device 104 and the host device 108, and as a result, the user device 104 can remotely control the gameplay session running on the host device 108 using one or more user interfaces (not shown) generated on the user device 104.

[0015] In one embodiment, if user information associated with user device 104 is not found in the user database 110, the application system 102 may identify user device 104 as an unregistered user device 104. Once such an unregistered user device 104 is identified, the application system 102 is configured to send registration information that enables user device 104 to register with the application system 102 in order to access, for example, one or more applications from the application database 112. The registration information may include, for example, pricing information, required user information, and registration validity. Once user device 104 has registered with the application system 102, user device 102 may request the application system 102 to run one or more applications that user device 104 has registered and / or other applications that register other applications.

[0016] As described above, in response to a request to execute an application received from a predetermined user device 104, the application system 102 identifies a host device 108 to fulfill the request. In one embodiment, the host device 108 is identified as suitable to fulfill the request based at least in part on a plurality of computing resources found to be available on the host device 108. Returning to the implementation of the cloud gaming application, if it is determined that sufficient computing resources such as memory, storage, graphics, and processing power are available on the host device 108 to execute the gameplay session, the host device 108 is selected from a plurality of host devices 108A to 108N to execute the requested gameplay session. In one embodiment, the plurality of host devices 108A to 108N include personal computers such as gaming rigs and / or dedicated computing devices for running high-performance applications. Other embodiments are also possible.

[0017] In one embodiment, the performance of the host device 108, particularly the performance required to determine whether the host device 108 is suitable for hosting a requested application session, is determined at least in part on a set of application-specific performance metrics. In some examples, these metrics may include streaming metrics such as frame rate, frame rate consistency, input latency (e.g., delay between local user input and screen rendering), network outbound bandwidth, and network latency, encoding capability such as encoded frames per second, and frame drop rate. These metrics may be affected by the capabilities of the host device 108's hardware components, such as the CPU, GPU, memory, storage I / O, and network adapter. Furthermore, these metrics may also be affected by factors such as virtualization overhead, the computational load of other background tasks on the host device 108, the network conditions between the host device 108 and the client device 104, and the ISP quality of service. These and other intended metrics are considered by the application system 102 when determining whether the host device 108 is capable of hosting an application session and providing adequate performance while doing so.

[0018] In one embodiment, the application system 102 is configured to generate an interface on the user device 104, which in turn renders a set of controls for controlling an application session running on the host device 108. When the user device 104 interacts with one or more controls rendered on the interface, the host device 108 detects the interaction via the P2P connection 130 and modifies the application session based on the detected interaction. In this way, the user device 104 can remotely interact with the application session hosted on the host device 108 as if it were running locally on the user device 104.

[0019] In another embodiment, when an ongoing application session hosted on a given host device 108 ends, the application system 102 is configured to extract application data generated as a result of the execution of the application session from the host device 108. The application system 102 may store the application data in a specific data location, for example, a user database 110, where the application data represents the state it was in at the end of the application session. Furthermore, in response to a request received from a user device 104 to restore the application session from its last stored state, the application system 102 may identify another host device 108 to fulfill the request. The restoration of the application session is shown in more detail in Figure 3.

[0020] If two or more host devices 108 are identified as candidates for hosting an application session, the application system 102 is configured to select the host device 108 that is geographically closest to the user device 104. In some cases, a given host device 108 may not have the capacity / bandwidth necessary to host the application, or it may be overused. In such cases, the application system 102 continues to search for a suitable host device 108 that is geographically closest to the user device 104 until a predetermined period of time has elapsed. The user's geographical location may be determined through various mechanisms such as GPS, IP address, and ping information. In one embodiment, once selected, the application system 102 sends a signal to the selected host device 108 to preload the requested application in order to minimize latency while the user device 104 interacts with the application.

[0021] In one embodiment, each host device 108 that hosts an application session is incentivized at least in part based on the number of application sessions that are determined to have been successfully hosted. In one example, an application session that is successfully hosted can be defined as an application session that is hosted within a predetermined latency limit and without interruption. According to this implementation, a given host device 108 is prioritized with respect to selection for hosting future application sessions at least in part based on the number of application sessions that have been successfully hosted. In another embodiment, to incentivize host device 108, application system 102 is configured to provide host device 108 with rewards including, but not limited to, discount coupons, monetary rewards, free subscriptions, etc. In one embodiment, account-based virtual currency and / or blockchain-based smart contracts can be generated and redeemed for both monetary and in-kind rewards for host device 108. For example, these rewards can include transferable play time on a cloud gaming service as a client, discounts on product purchases, exclusive early access to new product releases, exclusive badges for social media, or cosmetic assets, etc.

[0022] In one embodiment, the host device 108 is used to remotely host application sessions, replacing the server farms and data centers required in conventional cloud-based applications. Cloud-based applications, particularly cloud gaming, require substantial infrastructure to function properly, including data centers and server farms for running the games, as well as a high-speed internet connection with minimal latency for transmitting gameplay streams to users. This infrastructure may not be available or accessible to consumers, making cloud gaming impractical and costly in most areas. Furthermore, latency is a critical factor affecting the quality of cloud gaming services, as it can impact gameplay by introducing a delay between user inputs and their effects, especially in fast-paced games that rely on precise input. Therefore, by replacing data centers and server farms with host devices as described above, it is possible to provide requesting user devices with a high-quality, seamless experience in their gameplay without the need to invest in expensive dedicated hardware and software. Moreover, the crowdsourced remote gaming infrastructure can also reduce the infrastructure costs for service providers by offloading them to end users who agree to be remote hosts. Even considering the incentive costs of these host devices, the solutions presented herein provide service providers with a more flexible alternative to traditional cloud gaming implementations. Good performance may be achievable without significant investment in first-party servers when crowdsourced host and client clusters are located in the same location. Furthermore, these solutions can be used in conjunction with traditional cloud gaming service deployments to provide buffers or downtime gaps, while adjustments to the cloud infrastructure deployment are made to address shifts in the geographical location of the user base (as detailed in Figure 7).

[0023] Referring now to FIG. 2, a block diagram of an exemplary embodiment of various components of the application system 202 is shown. As illustrated, the application system 202 includes one or more interfaces 204, a memory 206, a processor 208, and a web server 210. In one embodiment, the one or more user interfaces 204 are configured to display data generated as a result of the processor 208 executing one or more programming instructions stored in the memory 206. In one embodiment, the processor 208 includes a plurality of cores configured to execute instructions. In some embodiments, the processor 208 further includes circuitry configured to perform parallel processing. In some embodiments, the processor 208 is a system-on-chip (SOC) that includes a plurality of hardware components (e.g., a CPU, a GPU, a memory controller, etc.). A plurality of such embodiments are possible and contemplated. For example, as illustrated, the processor 208 includes a connection circuit 212 and a selection circuit 214 for performing one or more functions described herein.

[0024] In one embodiment, the connection circuit 212 receives requests from the client device 216 for the execution of an application. According to this implementation, the client device 216 can request the application system 202 to execute a given application if the client device 216 does not have the computing resources for locally executing the application. The request is received by the application system 202 via a network 230 such as the Internet. For example, the client device 216 can request the application system 202 to access a cloud gaming site to execute one or more games that require hardware and software infrastructure for operation and that are not locally available to the client device 216. The required hardware and software infrastructure can include, for example, RAM capacity, storage capacity, processing power, operating system capabilities, etc.

[0025] In one embodiment, upon receiving a request from a client device 216 to run an application, the connection circuit 212 can determine whether the client device 216 is permitted to access the requested application. In one example, a client device registered with the application system 202 can access one or more applications stored in the application database 220. In one embodiment, the web server 210 processes requests received from one or more client devices, such as the client device 216, via the network 230. In some embodiments, the web server 210 processes requests to access one or more applications and responds to those requests, for example, by facilitating the authorization of login credentials for the client device 216. In one example, for a cloud gaming application, the web server 210 provides access to one or more user interfaces, which may take the form of one or more web pages, when the client device 216 accesses the application system 202 via a web browser. Furthermore, the web server 210 is configured to access user data stored in the user database 222 in order to provide these services. User data may include, but is not limited to, information such as username, password, registration date, email address, telephone number, subscription ID, application usage history, social graph information, application ownership, preferences, and other settings. However, if the client device 216 is unregistered, the connection circuit 212 provides an option to register the client device 216. The user of the client device 216 can register the client device 216 by entering the necessary registration information. In one embodiment, the connection circuit 212 may identify whether the client device is a registered or unregistered device based at least in part on user data stored in the user database 222.

[0026] Once it is determined that client device 216 is, for example, a registered device authorized by web server 210 (or an unregistered client device completes registration), the selection circuit 214 identifies a suitable host device 218 for running the application selected by client device 216. In one embodiment, the selection circuit 214 identifies host device 218 as suitable for hosting the requested application, at least in part, based on whether host device 218 required computing resources to run the application. For example, if the requested application is a cloud gaming application, the host device can be selected based on the availability of computing resources, such as graphics processing power, that are unavailable to client device 216. In another embodiment, the selection circuit 214 is configured to select a given host device from a plurality of host devices based on previous application sessions successfully run on that given host device. For example, the selection circuit 214 may determine that host device 218 has hosted the largest number of previously successful sessions for the selected application compared to other host devices. In yet another embodiment, host device 218 can also be selected based on its proximity to client device 216. For example, the selection circuit 214 can select the host device 218 if it is the closest to the client device 216, based on the geographical location of the client device. In other embodiments, for example, a combination of the factors described above can be used to select a given host device, and these embodiments are intended.

[0027] As shown in the figure, the host device 218 comprises a display 242, a controller 244, and one or more application resources 246. Similarly, the client device 216 comprises a display 252 and a controller 254. In one embodiment, the application system 202 generates a network connection between the host device 218 and the client device 216, for example, a peer-to-peer (P2P) connection 250. Using the P2P connection 250, an application session 260 hosted on the host device 218 can be controlled by the client device 216 as if the application session 260 were running locally on the client device 216. In one embodiment, the controller 244 is configured to execute an application session using one or more application resources 246 in response to commands received from the controller 254. Furthermore, the output generated by the execution is provided to the client device 216 by the host device 218 by rendering one or more user interfaces 262 on the client device 216's display 252. As shown in the figure, one or more user interfaces 262 are generated in real time on the display 252 of the client device 216, enabling access to the application session via the client device 216.

[0028] In one example, in a cloud gaming embodiment, the host device 218 can run a selected gameplay session and generate raw (uncompressed) video and audio. The video and audio are captured and encoded by the controller 244 for streaming purposes. In one embodiment, encoding can provide compression of the video and audio streams to reduce bandwidth usage and improve the gaming experience. Examples of encoding formats include H.265 / MPEG-H, H.264 / MPEG-4, H.263 / MPEG-4, H.262 / MPEG-2, WMV, VP6 / 7 / 8 / 9, etc. The encoded audio and encoded video are further packetized into network packets by the controller 244 for transmission over the P2P connection 250. In some embodiments, the host device 218 can further generate haptic feedback data, which may be packaged into network packets for transmission to the client device 216.

[0029] In one embodiment, the client device 216 decodes or reassembles audio packets, video packets, and haptic feedback packets to generate encoded audio, encoded video, and haptic feedback data. If the data is encrypted, the network packets are decrypted. The client device 216 then decodes the encoded audio and encoded video to generate raw audio and video data for display on the display device 252. The haptic feedback data may be processed to generate haptic feedback effects on the controller device 254 or another haptic interface device. An example of a haptic effect is vibration of the controller 254, such as a game controller.

[0030] Therefore, when an application hosted by the host device 218 responds to user input, the same process as described above can be performed to send and process the user input from the client device 216 to the host device 218. In one example, input data is generated by the user operating the controller 254. This input data is packaged as input data packets for transfer to the host device 218. The host device 218 is configured to pack and reassemble these packets to define the input data on the host device side and update the application state.

[0031] In one embodiment, the application system 202 is configured to monitor the transmission quality of the P2P connection 250 to ensure the quality of service of the application during the transmission of various packets. Furthermore, network conditions such as upstream and downstream network bandwidth may be monitored by the application system 202 and used to adjust application execution in response to changes in available bandwidth. In other words, the encoding and decoding of network packets may be controlled based on the current network conditions.

[0032] When an application session ends and / or the session completes execution, the application system 202 extracts all data generated as a result of the application's execution from the host device 218 and stores that data in the user database 222. In one embodiment, the application system 202 programs the host device 218 to host any given application so that the end user of the host device 218 cannot interact with the application while it is running on the host device 218. This may be done to ensure that the privacy of the client device 216 is maintained and that undesirable changes to the application execution are avoided. The application system 202 may allow the host device 218 to host the application using one or more containers or virtual machines so that the host device 218 can host the application in the background while the host device 218's local system can operate simultaneously without any modifications. This is illustrated in more detail in Figure 4.

[0033] Referring to Figure 3, a block diagram illustrating an exemplary implementation of session restoration for a saved application session is shown. As described above, the application system (e.g., application system 330) selects a host device to host the application requested by the client device. The host device runs the application session using local computing resources, and the network connection established between the client device and the host device allows the client device to control the application session locally. When the application session ends, the application system 330 deletes all data generated during the execution of the application session and stores it in a data store. Furthermore, when the client device requests restoration of the application session, the application system 330 uses the stored data to restore the application session on another host device.

[0034] As shown in Figure 3, user 302 using user device 304 is provided with a peer-to-peer network connection to access one or more applications hosted on a first host device 308 belonging to another user 310. The P2P connection is symbolized using a “game controller icon” 306, as shown in the figure. In one embodiment, the one or more applications hosted on host device 308 include a cloud gaming application. Furthermore, the peer-to-peer connection 306 allows user device 304 to control a gameplay session of the game application, if it is hosted on host device 308. As gameplay progresses, host device 308 continues to provide local computing resources to ensure that user device 304 can control the gameplay. In one embodiment, application system 330 is configured to monitor the P2P connection 306 between devices to ensure that the desired quality of service is maintained.

[0035] When user 302 ends a gameplay session, a command is sent from user device 304 to application system 330. Upon receiving the request, application system 330 extracts all application data generated as a result of the gameplay session from host device 308 and uploads the application data to data store 314. This is symbolized using a “save icon” 316, as shown in the figure. In one embodiment, the application data represents the state saved at the end of the gameplay session. Furthermore, the application data is uploaded to data store 314 and associated with user information previously stored for user 302. In one example, the user information includes username, password, registration date, email address, phone number, subscription ID, application usage history, social graph information, application ownership, preferences, and other settings of user 302.

[0036] In one embodiment, if user 302 wishes to restore a gameplay session from its last saved state, user device 304 sends a request to the application system 330 to restore the gameplay session. In one example, the restore request is generated when user 302 selects the “Restore Last Session” option shown on the display of user device 304. This option may be displayed, for example, each time user device 304 reconnects to the application system 330, in addition to one or more other options for accessing the game application. Other options may include starting a new game, viewing a record of a previous gameplay session, changing gameplay settings, or browsing leaderboards.

[0037] The application system 330 is configured to select a second host device, for example, a host device 320 belonging to user 322, to host the restored application session. In one embodiment, the host device 320 is selected from a plurality of host devices based at least in part on the number of application sessions hosted by the host device 320 that are determined to be successful. In another embodiment, the host device 320 may also be selected based on its geographical proximity to user device 304. In some embodiments, a combination of these factors is used to select the host device 320.

[0038] In one embodiment, an application session is restored from its last saved state hosted on the host device 320, using application data stored in the data store 314 so as to be cross-referenced with user information previously stored for user 302. In other words, the application system 330 identifies the relevant application data stored in the data store 314 based on its association with user information for user 302. If no such association is found, the application system 330 sends an error message to the user device 304. Cases where no association exists between stored application data and user information may occur due to improper termination of the last session, corruption of user information, expiration of the user account, etc. Other embodiments are also possible.

[0039] Once the application data is used to identify the last saved state of gameplay, the application system 330 downloads the application data to the selected host device 320. This is referred to in the figure by the “save icon” 332. Furthermore, the application system 330 establishes a P2P connection (indicated by the “game controller” icon 336). Using the P2P connection 336, the user device 304 can restore gameplay from the last saved state. While Figure 3 illustrates the restoration of a gameplay session, in alternative embodiments, the described technique can be used for any number of cloud-based applications. Such embodiments are contemplated.

[0040] Referring now to Figure 4, several embodiments for running an application on a host device are shown. As shown in Figure 4, the host device 402 hosts an application session 406 requested by the client device 404. As described above, the application session 406 runs on the host device 402 using computing resources local to the host device 402. In one embodiment, the application session 406 is hosted on the host device 402 in such a way that users using the host device 402 are restricted from modifying the application session while it is being hosted. In one example, the user of the host device 402 can only turn off the host device 402 and cannot interact with the application session 406 in any other way.

[0041] In one embodiment, process isolation 408 is performed to isolate the application session 406 from other programs running on the host device 402, so that the application session 406 runs as a background process separate from the host operating system 420. As described with respect to Figures 4A to 4C, process isolation 408 may be performed using one or more of sandboxing or containerization. Furthermore, a user interface (e.g., a streaming application interface 410) is generated, and as a result, application data 412 is used to relay the application session 406 to a remote client device 404, as shown. In one embodiment, application data 412 is generated and used to stream the application session 406 to the remote client device 404, for example, based on the User Datagram Protocol (UDP), which is primarily used to establish connections between applications over the Internet, with a focus on minimizing latency and tolerating data loss. Other embodiments are described with respect to Figures 4A to 4C.

[0042] Referring next to Figure 4A, a sandbox 414 is used to isolate the execution of the application session 406. In one embodiment, the sandbox 414 allows a user of the host device 402 to run the application session 406 in a secure and isolated environment. For example, using the sandbox 414, the host device 402 can create a temporary virtual machine, which can be discarded when the application session 406 terminates. The sandbox 414 provides a secure way to host the application session without risking modification of the application session by the host operating system 420 while the application session is running. Note that, unless otherwise specified, similar symbols throughout the description of Figures 4A to 4C indicate the same system components.

[0043] In one embodiment, a software development kit (SDK) is used to build and deploy an SDK server 416 that enables real-time streaming of application data generated as a result of the execution of an application session 406 over a network, such as the Internet. The streaming SDK server 416 provides tools and APIs for generating and customizing streaming data, including, for example, video and audio codecs, transcoding, and adaptive bitrate streaming. The server 416 can be integrated into a website or application, enabling remote client devices 404 to access streaming media content in real time.

[0044] In one embodiment, a UDP relay 418 is used to forward or proxy UDP packets between two network endpoints, namely an SDK server 416 and a remote client device 404. This is typically done when there is a network device such as a firewall or router that blocks or restricts UDP traffic between the two endpoints. The relay 418 receives UDP packets from one endpoint, for example from the SDK server 416, and retransmits them to the other endpoint, namely the remote client device 404, and vice versa. This technique can be used in a variety of applications where low latency and real-time data transmission are important, including online gaming, VoIP, and multimedia streaming.

[0045] Referring here to Figure 4B, container 422 may be used to isolate an application session 406 hosted on host device 402. In one embodiment, container 422 enables a software abstraction that bundles the application session's code and its dependencies into a single package. Unlike virtual machines (VMs), multiple containers can run on the same machine, e.g., host device 402, and share the underlying operating system kernel with other containers, each running in its own isolated process within user space. In one embodiment, a container runtime 424 is used to manage the operation of container 422. The container runtime is a software component that enables the creation, management, and execution of containers. In one example, the container runtime 424 is responsible for starting and stopping container 422, managing the container 422's lifecycle, and providing a user interface for interacting with the host operating system 420. The container runtime 424 can use a container engine (not shown) to package the application code and its dependencies into a container image, and then deploy and run the container image on various container platforms. Furthermore, the container runtime 424 provides security features such as user and namespace isolation to ensure that container 422 is isolated from other containers and from the host operating system 420.

[0046] Figure 4C illustrates yet another implementation for process isolation of an application session 406 using a virtual machine (VM) 442. In one embodiment, the application session 406 running on the host device 402 renders the relevant video frames into a virtual frame buffer (VFB) 440. Furthermore, the streaming SDK server 416 reads the video frames from the VFB 440, encodes them, and sends them to a remote client 404 over a network (e.g., a peer-to-peer network connection). In one embodiment, an image management service 444 is configured for setting up and dismantling the application session 406, restoring and saving data extracted from the host device 402, and managing configuration files, etc. In an alternative embodiment, VM 442 can be replaced by a container such as Docker to leverage out-of-box image management functionality. For this purpose, the container may handle software abstractions instead of VM 442.

[0047] Referring to Figure 5, a method for hosting an application session on a host device using a peer-to-peer network is illustrated. As described above, the client device requests the application system to run the desired application session. The application system selects a host device to host the application session and creates a peer-to-peer (P2P) link between the client device and the host device. The client device can then use the P2P link to control the application session locally.

[0048] In one embodiment, an application device receives a connection request from a client device (block 502). The connection request indicates a request to connect the client device to the application so that the client device can interact with the application's session. In one example, the application includes a cloud gaming site, and the connection request may indicate a request to run a gameplay session for a selected game.

[0049] The application system, upon receiving a connection request, searches for a host device to fulfill the request (block 504). In one embodiment, the host device is searched for based at least in part on a previously successful application session by the given host device, the geographical proximity of the host device to the client device, and other factors. The application system then determines whether such a host device has been found (conditional block 506). If no such host device is identified, or if an identified host device is unavailable (conditional block 506: "no"), the application system further determines whether the timeout period has elapsed (conditional block 508). In one example, a timeout occurs if the application system fails to find a host device within a specified period after receiving the connection request. For example, a timeout error occurs if the application system is unable to process and respond to the connection request within a set time limit due to high traffic or an overload condition. If the timeout period has elapsed (conditional block 508: "yes"), the application system sends a timeout error to the client device (block 510). Otherwise, the application system continues searching for a suitable host device (conditional block 508: "no").

[0050] If a host device is found (conditional block 506: "yes"), the application system creates a P2P link between the client device and the host device (block 512). Furthermore, the application session is initiated to be hosted on the host device (block 514). In one embodiment, the application session is hosted on the host device in such a way that users using the host device are restricted from viewing and / or modifying the execution of the application session. This is done using a virtual machine or container environment (as shown in Figure 4).

[0051] The application system provides a set of application controls to the client device when the client device is hosted on the host device (block 516). In one embodiment, the application system renders a user interface on the client device's display and provides a set of controls through the user interface. The controls are provided to the client device so that it can access and control the application session locally in real time when the application session is hosted on the host device. Furthermore, outputs generated on the host device in response to the client device's interaction with the controls are also relayed to the client device in real time. In this way, the client device can access the application as if it were running locally.

[0052] A client device can terminate an application session at any point during its execution. In one embodiment, when a client device terminates an application session, an indicator of the termination of the application session is received by the application system (block 520). Upon receiving the indicator, the application system extracts all data generated as a result of the execution of the application session from the host device (block 522). Furthermore, the extracted data is stored in a data store and the P2P connection is disconnected (block 524). In one embodiment, the extracted data is associated with the client device's user's previous user information, which is previously stored in the data store. The association between the stored data and the user information is used to restore the stored application session, as shown in Figures 3 and 6. In another embodiment, the user information is also used to determine one or more applications that the client device is permitted to access.

[0053] Figure 6 shows an exemplary method for restoring a saved application session using a host device. In one embodiment, when an application session running on a host device terminates, the application system saves the state of the application session at the time of termination. This is referred to herein as the last saved state of the application session. For example, if the application is a game application, the last saved state may represent one or more of the following: the last level achieved, the accumulated points at the time of termination, the number of turns remaining at the time of termination, etc. When a client device requests the restoration of an application session from its last saved state, the application system identifies a host device for restoring the application system. The application system is then restored using previously saved data and user information.

[0054] As shown in the figure, the application system receives a session restore request from the client device (block 602). In one embodiment, the session restore request includes at least a predetermined endpoint from which the application session previously ended and from which restoration is desired. In response to receiving the restore request, the application system searches for a suitable host device to restore the application session (block 604). As described above, the host device is searched for based at least in part on the application session previously successfully completed by the given host device, the geographical proximity of the host device to the client device, and other factors.

[0055] The application system then determines whether such a host device has been found (conditional block 606). If no such host device is identified, or if an identified host device is unavailable (conditional block 606: "no"), the application system further determines whether the timeout period has elapsed (conditional block 608). For example, a timeout may occur if the application system fails to find a host device within a specified period after receiving a restore request. If the timeout period has elapsed (conditional block 608: "yes"), the application system sends a timeout error to the client device (block 610). Otherwise, the application system continues searching for a suitable host device (conditional block 608: "no"). Furthermore, if a host device is found (conditional block 606: "yes"), the application system creates a P2P link between the client device and the host device (block 612).

[0056] The application system restores the application system on the host device using stored application session data, which is cross-referenced with previously stored user information associated with the client device (block 614). The application system then provides multiple application controls to the client device, for example, by rendering an interface on the client device's display and providing multiple controls through the interface (block 616). In one embodiment, based on settings and preferences indicated by user information, the application system determines the commands for running the restored application session. For example, if the application is a game application, each time a session is restored, the restored session may run in single-player mode until multiplayer mode is activated. In another example, each time a session is restored, one or more controls may be set to default based on previously stored user preferences until a change is made to the controls. Other embodiments are also possible.

[0057] Figure 7 shows an exemplary implementation of a computing environment that enables multiple users to access cloud video games over a network. As illustrated, the game console 702 runs a gameplay session 704. In one embodiment, the game console 702 may be located within a data center 706 alongside other similar machines capable of running cloud video gameplay sessions for different client devices. Note that the game console 702 can take different forms in various embodiments, including a game console, bladed game console hardware, a server computer / blade, a virtual machine running on hardware resources, or any other hardware setup capable of providing a suitable execution environment for cloud video games.

[0058] In the illustrated embodiment, a video game play session is defined for a primary client device 708. The game play session is streamed to the primary client device 708 associated with a given user via a network 710, which may include the Internet. More specifically, the video game play session instantiates and maintains a game state that is updated based on received input commands. The game console 702 renders video frames and audio based on the game state and defines a video stream (including both video and audio data) that is transmitted to the primary user device via the network 710. As used in this disclosure, and unless otherwise noted or otherwise evident from this disclosure, video refers to both video data and corresponding audio data. The client device 708 renders the video stream to a user interface (not shown) generated on a display 712 for the user to view. The user can interact with the video game through the operation of a controller device 714, which may be any device that the primary user can use to provide interactive input to the video game, such as a game controller, peripherals, keyboard, mouse, touchpad, trackball, motion controller, video camera, depth camera, microphone, etc.

[0059] In the embodiments described above, the game console 702 transmits a video stream to the client device 708. In certain embodiments, a streaming server 716 is used to manage the streaming of the video stream to the client device 708. The streaming server 716 is configured to process the raw video output generated by the game console 702, which may include encoding the raw video output in a compressed format to generate a video stream, transcoding the raw video output from one format / codec to another, or adjusting the bitrate of the raw video output based on various factors, such as network bandwidth, client device hardware capabilities, device application / browser type, etc. In some embodiments, the streaming server 716 is located within the data center 706, and in other embodiments, the streaming server 716 may be located elsewhere, such as in another data center. Furthermore, in some embodiments, the streaming server 716 is run by the game console 702.

[0060] In some embodiments, if the data center 706 and / or streaming server 716 are overloaded, facing downtime, or otherwise unavailable for processing game applications for client device 708, the application system 702 is configured to identify a host device 718 to host a game application session for client device 708. For example, client device 708 can send a request to the application system 702 to connect to a cloud game application and access a desired game title. In response to the request, the application system 702 identifies the user associated with client device 708 by accessing user account information stored in a user data store, such as a user database 720. The application system 702 verifies the identified user to determine whether client device 708 is permitted to access the requested game title. In one embodiment, the application system 702 interacts with the application database 730 to determine one or more game titles that client device 708 is permitted to access. If the application system 708 determines that the game title requested by the client device 708 is accessible, it instructs the host device 718 to remotely host a gameplay session of the game. Furthermore, a network connection, such as a P2P network connection 740, is established between the client device 708 and the host device 718 so that the client device 708 can remotely control the gameplay session running on the host device 718. In one embodiment, the client device 708 can remotely control the gameplay session via a controller device 714.

[0061] In one embodiment, using host devices in addition to data centers to host gameplay sessions can provide a buffer or “stop-gap” solution when host data centers and streaming servers are unavailable and adjustments are made to the cloud infrastructure deployment. Furthermore, the methods and systems described herein can also function as a solution to address shifts in the geographical location of the client base, as relocating traditional cloud gaming infrastructure to meet demand can be expensive for service providers. The solutions presented herein may give service providers a more flexible alternative to traditional cloud gaming implementations. When a group of crowdsourced hosts and cloud gaming clients are located in the same location, good performance may be achievable without high investment in first-party servers. While Figure 7 illustrates a gameplay session, it is worth noting that in alternative embodiments, the described technology can be used for any number of cloud-based applications. Such embodiments are contemplated.

[0062] It should be emphasized that the embodiments described above are merely non-limiting examples of embodiments. A large number of variations and modifications will become apparent to those skilled in the art once the above disclosure is fully understood. The following claims are intended to be construed as encompassing all such variations and modifications.

Claims

1. It is a method, The application system receives a first request from a first network-connected device to initiate an application session. The application system identifies a second network-connected device that satisfies the first requirement, The application system starts executing the application session on the second network connection device, The application system includes generating a plurality of controls for the first network-connected device to control the application session running on the second network-connected device, method.

2. The second network-connected device is identified from a plurality of host devices, and the identification of the second network-connected device is performed at least in part on the computing resources available in the second network-connected device. The method according to claim 1.

3. In response to a termination request to end the execution of the aforementioned application session, the application data generated during the execution of the aforementioned application session is extracted from the second network-connected device. This includes storing the aforementioned application data in a database, The method according to claim 1.

4. Receiving a second request from the first network-connected device, which is different from the first request, wherein the second request includes a request to restore the application session. Identifying a third network-connected device that satisfies the second requirement, This includes restoring the execution of the application session on the third network-connected device based at least partially on the application data, The method according to claim 3.

5. This includes generating an interface in the second network-connected device, the interface rendering the plurality of controls to control the application session running on the second network-connected device, The method according to claim 1.

6. This includes identifying the number of application sessions hosted by the second network-connected device that were determined to be successful, The method according to claim 1.

7. Based at least in part on the aforementioned number, the selection of the second network-connected device from among multiple host devices to serve one or more future requests relating to the execution of an application session includes: The method according to claim 6.

8. This includes calculating one or more rewards for the second network-connected device based at least in part on the number of successful application sessions performed using the second network-connected device. The method according to claim 6.

9. It is a system, Equipped with a processor, The aforementioned processor, Receiving a first request from a first network-connected device to initiate an application session, Identifying a second network-connected device that satisfies the first requirement, Starting the execution of the application session on the second network-connected device, For the first network connection device, a plurality of controls for controlling the application session running on the second network connection device, It is configured to do the following: system.

10. The second network-connected device is identified from a plurality of host devices, and the identification of the second network-connected device is performed at least in part on the computing resources available in the second network-connected device. The system according to claim 9.

11. The aforementioned processor, In response to a termination request to end the execution of the aforementioned application session, the application data generated during the execution of the aforementioned application session is extracted from the second network-connected device. The aforementioned application data is stored in a database, It is configured to do the following: The system according to claim 9.

12. The aforementioned processor, Receiving a second request from the first network-connected device, which is different from the first request, wherein the second request includes a request to restore the application session. Identifying a third network-connected device that satisfies the second requirement, Restoring the execution of the application session on the third network-connected device based at least partially on the saved application data, It is configured to do the following: The system according to claim 11.

13. The processor is configured to generate an interface on the second network-connected device, and the interface renders the plurality of controls to control the application session running on the second network-connected device. The system according to claim 9.

14. The processor is configured to identify the number of application sessions hosted by the second network-connected device that are determined to be successful. The system according to claim 9.

15. The processor is configured to select the second network-connected device from among a plurality of host devices to serve one or more future requests relating to the execution of an application session, at least in part based on the number. The system according to claim 14.

16. The processor is configured to calculate one or more rewards for the second network-connected device based at least in part on the number of successful application sessions performed using the second network-connected device. The system according to claim 14.

17. A computing system, At least one interface, Memory and Equipped with a processor, The aforementioned processor, Receiving a first request from a first network-connected device to initiate an application session, Identifying a second network-connected device that satisfies the first requirement, Starting the execution of the application session on the second network-connected device, For the first network connection device, a plurality of controls for controlling the application session running on the second network connection device, It is configured to do the following: system.

18. The second network-connected device is identified from a plurality of host devices, and the identification of the second network-connected device is performed at least in part on the computing resources available in the second network-connected device. The system according to claim 17.

19. The aforementioned processor, In response to a termination request to end the execution of the aforementioned application session, the application data generated during the execution of the aforementioned application session is extracted from the second network-connected device. The aforementioned application data is stored in a database, It is configured to do the following: The system according to claim 17.

20. The aforementioned processor, Receiving a second request from the first network-connected device, which is different from the first request, wherein the second request includes a request to restore the application session. Identifying a third network-connected device that satisfies the second requirement, Restoring the execution of the application session on the third network-connected device based at least partially on the saved application data, It is configured to do the following: The system according to claim 19.

Images