High-speed save data storage for cloud gaming

Abstract

To provide high-speed save-data storage for video games to be executed on a cloud gaming server.SOLUTION: A data access request issued by the video game is received by a management server. The data access request identifies requested user data stored in the cloud storage server. The requested user data is copied from a lower performance data storage device within the cloud storage server into a high-speed data storage device within the management server. The high-speed data storage device operates at a higher data rate and a lower latency than the data storage device within the cloud storage server. The high-speed data storage device in the management server is used to fulfill the data access request by the video game instead of using the lower performance data storage device in the cloud storage server to fulfill the data access request.SELECTED DRAWING: Figure 3

Description

[Background technology]

[0001] The video game industry has undergone many changes over the years. As computing power has increased, video game developers have also developed game software that takes advantage of this increased computing power. To do so, video game developers have coded games that incorporate advanced arithmetic and mathematics to create a highly realistic gaming experience.

[0002] Game developers are not only responding to the expansion of computing power, but are also developing online games that enhance the interaction or dialogue between users and gaming systems. Interactive online games, or interactive online games, provide users with the ability to play interactively against or with other users over the Internet and capture such interactions or dialogue. Furthermore, with the growing trend toward cloud-based computing, game developers are developing new ways to play existing legacy games and develop new games by taking advantage of increased computing power and increased user interactivity. Cloud-based computing systems can include cloud-based processing servers in communication with remote client systems residing at game players' locations, which can be configured to run game applications, receive input from game players, and render images on a display for the game players to view. It is against this background that the present invention was made. Summary of the Invention

[0003] In one exemplary embodiment, a cloud gaming system is disclosed. The cloud gaming system includes a cloud gaming server configured to execute a video game according to input received from a user's controller device. The cloud gaming server is configured to direct the transmission of a video stream reflecting the user's play of the video game to the user's local computing system. The cloud gaming system also includes a cloud storage server. The cloud storage server includes a data storage device for storing user data of the user. The user data includes data that records or memorializes the user's play of the video game. The cloud gaming system also includes an administration server in data communication with both the cloud gaming server and the cloud storage server. The administration server includes a high-speed data storage device that operates at a higher data rate and lower latency than the data storage device in the cloud storage server. The administration server is programmed to receive a data access request issued by a video game running on the cloud gaming server. The data access request identifies the requested data stored in the data storage device in the cloud storage server. The administration server is programmed to copy the requested data from the data storage device in the cloud storage server to the high-speed data storage device. The management server is programmed to respond to data access requests using the high-speed data storage device instead of the cloud storage server responding to the data access requests.

[0004] In an exemplary embodiment, a method for operating a cloud gaming system is disclosed. The method includes executing a video game on a cloud gaming server according to input received from a user's controller device. The method includes operating the cloud gaming server to direct transmission of a video stream reflecting the user's play of the video game to the user's local computing system. The method also includes storing user data of the user on a cloud storage server. The user data includes data recording the user's play of the video game. The method also includes receiving a data access request at an administrative server. The data access request is issued by a video game running on the cloud gaming server. The data access request identifies requested data stored on a data storage device in the cloud storage server. The method also includes copying the requested data from the data storage device in the cloud storage server to a high-speed data storage device in the administrative server. The high-speed data storage device operates at a higher data speed and lower latency than the data storage device in the cloud storage server. The method also includes responding to the data access request using the high-speed data storage device in the administrative server instead of the cloud storage server responding to the data access request. [Brief explanation of the drawings]

[0005] [Figure 1] 1 illustrates a cloud gaming system according to some embodiments. [Figure 2] 1 illustrates a process diagram of a method for triggering data transfer from a low-speed data storage device in a cloud storage server to a high-speed data storage device in a management server using a control path API call by the cloud gaming server before a data path API call by the cloud gaming server to access the transferred data, according to some embodiments. [Figure 3] 1 illustrates an exemplary diagram of some internal components of a cloud gaming server and an administrative server, according to some embodiments. [Figure 4] 1 illustrates a flowchart of a method for operating a cloud gaming system, according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0006] In the following detailed description, several specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it will be apparent to one skilled in the art that embodiments of the present disclosure may be practiced without some or all of these specific details. In other instances, well-known process operations have not been described in detail in order to not unnecessarily obscure the present disclosure.

[0007] Next-generation video game consoles will have extremely fast data storage solutions, such as Non-Volatile Memory Express (NVMe) solid-state drives (SSDs), which are much faster than currently available data storage solutions such as hard disk drives (HDDs). The extremely fast data storage solutions of next-generation video game consoles pose challenges for cloud gaming, which relies on shared storage servers for storing not only game saves but also game data and user data. In traditional cloud gaming systems, user data is typically stored within cloud storage solutions, such as cloud storage servers, which operate at HDD-level data storage performance to keep costs down. However, next-generation video game consoles will require data storage performance at least at the NVMe level to meet the data access expectations of game console titles (video games).

[0008] Disclosed herein are systems and methods for addressing the challenge of providing NVMe-level data storage performance for running online video games while continuing to use cost-effective HDD technology for storing user data related to the play of the online video games. It should be understood that the term online video game, as used herein, refers to a video game provided by a cloud computing system to a client computing device for play by a user on the client computing device. The online video games referred to herein may be single-player or multiplayer video games. The principle of the disclosed systems and methods is to provide close data communication between a cloud gaming server that runs the online video game and a cloud data storage solution in which data currently needed by the running online video game is stored in a high-speed data storage / system within a management server, while save data and other game-related data not urgently needed by the running online video game is stored in a slower, cost-effective cloud storage server.

[0009] FIG. 1 illustrates a cloud gaming system 100 according to some embodiments. The cloud gaming system 100 includes a cloud gaming server rack 101 that includes a plurality (N) of cloud gaming servers 103-1 through 103-N. In some embodiments, the number (N) of cloud gaming servers 103-1 through 103-N can be up to 100 or more. The cloud gaming server rack 101 also includes a plurality (X) of management servers 105-1 through 105-X and one or more storage servers 107. The storage servers 107 include a number of data storage devices 107A, such as HDDs and / or SSDs. The storage servers 107 store all video games available for play on the cloud gaming system 100. Each management server 105-1 through 105-X is responsible for managing cloud gaming sessions. Additionally, a given management server 105-1 through 105-X can be responsible for managing multiple cloud gaming sessions simultaneously. In some embodiments, each management server 105-1 through 105-X powers on one or more cloud game servers 103-1 through 103-N as needed, loads the requested video game from storage server 107 onto a predetermined one of cloud game servers 103-1 through 103-N, initiates execution of the requested video game on the predetermined one of cloud game servers 103-1 through 103-N, logs the user into the predetermined one of cloud game servers 103-1 through 103-N, and mounts user data 111 for the logged-in user from cloud storage server 109, among other operations.

[0010] In some embodiments, storage server 107 can be replaced by a storage configuration that transforms HDDs and / or SSDs into mini-servers by incorporating a network interface card (NIC) and a small central processing unit (CPU). An example of such a storage configuration is an embedded SSD (ESSD). For example, in some ESSDs, a memory controller integrated circuit (IC) chip is combined with an SSD in a single package using a ball grid array (BGA) format in a flip-chip packaging process. In some embodiments, the single package containing the IC chip and SSD can be mounted directly to a printed circuit board (motherboard) without using a module connector. Of course, it should be understood that there are many possible configurations of ESSDs that can be used in place of storage server 107 in various embodiments. One motivation for using ESSDs instead of storage server 107 is that traditional storage servers no longer scale well, as storage data communication becomes faster than network data communication. In some embodiments, when an ESSD storage configuration is used instead of storage server 107, a game can be striped across multiple ESSDs. In these embodiments, cloud gaming servers 103-1 through 103-N need to reach different drives (different ESSDs) for different parts of the game. Also, in some embodiments, the use of ESSDs can be extended to cloud storage server 109. In such an embodiment, it is necessary to have a cluster of many ESSDs, along with management servers 105-1 through 105-X, which serve to hide the implementation details of the ESSD-based cloud storage server 109 from cloud gaming servers 103-1 through 103-N.

[0011] In cloud gaming (or online gaming), user (game player) 121 operates local computing system 120 to log into cloud gaming system 100 over network 123, such as the Internet, to play online video games that are executed remotely on one or more of cloud gaming servers 103-1 through 103-N. In some embodiments, user 121 logs into cloud gaming system 100 through one of management servers 105-1 through 105-X. However, it should be understood that there are various cloud services associated with user 121 logging in. For example, there are cloud services involved in authenticating user 121, ensuring that user 121 has an Internet connection of sufficient quality, and finding an available server near user 121 that user 121 can log in to. In some embodiments, local computing system 120 includes a game controller 125 and a display device 126, such as a television or monitor. In some embodiments, local computing system 120 includes a dongle 128 connected to display device 126, the dongle 128 configured to enable data communication between game controller 125 and display device 126 such that a network interface controller (NIC) of display device 126 can be used to transmit signals from game controller 125 to network 123. In some embodiments, local computing system 120 also includes a local gaming console 127. In these embodiments, signals generated by game controller 125 can be transmitted to network 123 via local gaming console 127 using the NIC of local gaming console 127. Also, in some embodiments, game controller 125 can transmit signals directly to network 123 through a local area network, such as a WIFI network. Also, in some embodiments, user 121 can play online video games on a local computing device other than local computing system 120. For example, in some embodiments, user 121 can play online video games on a mobile computing device, such as a mobile phone, tablet, laptop, or smart car system, among others. Also, in some embodiments, user 121 can play online video games on a desktop computing system. For ease of explanation, reference herein to local computing system 120 also conveys a reference to any other computing device on which user 121 can play online video games.

[0012] Additionally, in various embodiments, data (such as video stream data or audio data or haptic feedback data) is communicated from cloud gaming system 100 to local computing system 120 via network 123. In some embodiments, data is communicated from cloud gaming system 100 to display device 126 via network 123. In some embodiments, data is communicated from cloud gaming system 100 to local gaming console 127 and from local gaming console 127 to display device 126 via network 123. Also, in some embodiments, data is communicated from cloud gaming system 100 to game controller 125 via network 123.

[0013] During game play, user 121 generates game commands that are transmitted to cloud game servers 103-1-103-N over network 123. In some embodiments, user 121 generates game commands using game controller 125, and the game commands are transmitted in the form of signals from game controller 125 to network 123 and then to cloud game servers 103-1-103-N. In some embodiments, the game commands are transmitted in the form of signals from game controller 125 to local game console 127 or display device 126, from which the game commands are transmitted over network 123 to cloud game servers 103-1-103-N. In various embodiments, transmission of game commands in the form of signals from game controller 125 to network 123 may occur via a wired or wireless network connection, or a combination thereof. Cloud game servers 103-1-103-N execute video games according to the game commands received from user 121 and generate video streams reflecting the user's play of the video game. The video streams are transmitted from cloud game servers 103-1-103-N over network 123 to local computing systems 120 of users 121 for display on display devices 126.

[0014] Cloud gaming servers 103-1-103-N load all of game data 113 and game executable files 115 from storage server 107 using network storage protocol 117. In some embodiments, network storage protocol 117 utilizes block storage such as iSCSI (Internal Small Computer System Interface), among others. In some embodiments, network storage protocol 117 utilizes file storage such as NFS (Network File System) or CIFS (Common Internet File System), among others. In some embodiments, cloud gaming servers 103-1-103-N may be limited in supported network storage protocols 117. For example, in some embodiments, cloud gaming servers 103-1-103-N may be limited in supported network storage protocols 117 due to a lack of software drivers. Or, in some embodiments, cloud gaming servers 103-1-103-N may be limited in supported network storage protocols 117 due to some network storage protocols utilizing excessive resources, such as CPU usage and / or memory usage. In some embodiments, cloud game servers 103-1-103-N may be limited in the network storage protocols 117 they support, so that access to user data 111 by cloud game servers 103-1-103-N may occur through management servers 105-1-105-X, with management servers 105-1-105-X acting as proxies. In these embodiments, management servers 105-1-105-X mount user data 111 from cloud storage server 109 and expose user data 111 to cloud game servers 103-1-103-N using a data storage protocol supported by cloud game servers 103-1-103-N.

[0015] In various embodiments, cloud storage server 109 exposes data storage via various data storage protocols, such as object storage, file storage, block storage, or other data storage protocols. Management servers 105-1 through 105-X are equipped to interface with any data storage protocol used by cloud storage server 109, whether it is object storage, file storage, block storage, or other data storage protocol. Management servers 105-1 through 105-X are also configured to expose their own data storage via various data storage protocols, such as object storage, file storage, block storage, or other data storage protocols. Additionally, the data storage protocol through which management servers 105-1-105-X expose their data storage may be the same as or different from the data storage protocol through which cloud storage server 109 exposes its data storage. For example, in some embodiments, management servers 105-1-105-X are programmed to mount object storage from cloud storage system 109 as a file system and then present the mounted file system to cloud game servers 103-1-103-N via an NFS protocol. Similarly, in another exemplary embodiment, management servers 105-1-105-X are programmed to mount block storage from cloud storage system 109 as a file system and then present the mounted file system to cloud game servers 103-1-103-N. In some embodiments, the block storage is based on Ceph RADOS (Reliable Autonomic Distributed Object Store) block devices. However, in other embodiments, the block storage can be based on any type of block data storage protocol.

[0016] Having the management servers 105-1-105-X act as proxies in the data path between the cloud game servers 103-1-103-N and the cloud storage server 109 may add some latency in fulfilling data access requests issued by the cloud game servers 103-1-103-N. However, having the management servers 105-1-105-X act as proxies in the data path between the cloud game servers 103-1-103-N and the cloud storage server 109 provides more flexibility in how data transfer is managed between the cloud storage server 109 and the cloud game servers 103-1-103-N. For example, in some embodiments, the management servers 105-1-105-X may operate to buffer data input / output requests and / or cache data as needed and issued by the cloud game servers 103-1-103-N. For example, if cloud storage server 109 is busy or not immediately available for some reason, management servers 105-1 to 105-X can absorb the lack of availability of cloud storage server 109 by buffering data input / output requests issued by cloud game servers 103-1 to 103-N.

[0017] Traditionally, the interactions between the cloud game servers 103-1-103-N and the management servers 105-1-105-X / cloud storage servers 109 are somewhat decoupled. For example, the cloud game servers 103-1-103-N are provided with a network file system, and some network storage software manages data read and write operations for the cloud game servers 103-1-103-N. In this situation, the management servers 105-1-105-X are simply responsible for mounting data from the cloud storage servers 109, but have no knowledge of what data is being read or written by or accessed by the video games running on the cloud game servers 103-1-103-N.

[0018] Next-generation game consoles will use high-speed NVMe data storage drives, operating at data rates of 4 to 5 gigabytes per second (GB / s). For comparison, the 2.5-inch HDDs currently used in cloud storage systems 109 peak at data rates of 100 megabytes per second (MB / s). HDDs also have much higher latency than NVMe data storage drives. For example, typical HDD latency is in the range of approximately 10 milliseconds to approximately 20 milliseconds, while typical NVMe data storage drives have a latency of less than approximately 0.1 milliseconds. As storage performance improves, so do expectations in video game programming regarding how quickly a video game can access data storage. Video games often make assumptions about data storage performance. If actual data storage performance differs significantly from the data storage performance assumed by the video game, the video game may crash or exhibit other undefined behavior.

[0019] In some embodiments, the same video games used / executed on local gaming consoles 127 are stored, often without modification, in storage server 107 and executed by cloud gaming servers 103-1-103-N in cloud gaming. These video games are developed using performance assumptions based on video games executed on local gaming consoles 127 with faster storage solutions, particularly with respect to data read and write operations. Thus, when the same video games are executed on cloud gaming servers 103-1-103-N, the performance of the data storage solutions implemented on cloud gaming servers 103-1-103-N substantially matches or exceeds the performance of the local data storage solutions available on local gaming consoles 127.

[0020] The data storage solutions implemented in the cloud gaming system 100 need to be updated to keep up with the expected data read and write performance of next-generation video games, as set by the extremely fast data storage solutions implemented in the next-generation local game consoles 127. In some embodiments, existing cloud storage servers 109 can be updated to use high-speed NVMe storage solutions. However, this approach is very expensive. Therefore, the goal of a balanced solution is to reduce costs by retaining data stored on performance-limited HDDs in the cloud storage servers 109, while ensuring that data currently needed by the video games is available through high-speed storage in the management servers 105-1 through 105-X. This goal can be achieved through close cooperation between cloud gaming servers 103-1-103-N and the cloud storage solution, which is contrary to prior art techniques that essentially decouple or separate the interactions between cloud gaming servers 103-1-103-N and management servers 105-1-105-X / cloud storage servers 109. In some embodiments, close cooperation between cloud gaming servers 103-1-103-N and the cloud storage solution involves hooking F into a file system API (application programming interface) that is called by video games running on cloud gaming servers 103-1-103-N to obtain information about data that the video games are currently accessing, data that the video games will be able to access next / soon, and when the video games will finish using particular data.

[0021] File system access APIs trigger data access to occur at specific times for specific reasons. Various operating systems conform to the POSIX (Portable Operating System Interface) standard, which defines APIs for many operating system concepts, including file system access. The four most common POSIX APIs for file system access are "open," "close," "read," and "write." The "open" API is used to open a file. The "close" API is used to close a file, triggering the writing of data back to the data storage device if the data has not already been written back to the data storage device. The "read" API is used to read data. The "write" API is used to write data. The "open" and "close" APIs are considered control path APIs, which are APIs used to gain access to a file or change permissions. The "read" and "write" APIs are considered data path APIs, which are APIs used to access or modify data.

[0022] In some embodiments, the video game console API utilizes the POSIX API but with some additions. A typical personal computer or server computer typically has a single data storage device, such as a HDD or SSD, with one or more partitions, each with a file system. The storage system of a video game console is configured in a similar manner to a personal computer or server computer. However, on a video game console, for security and user isolation reasons, save data and other user data are often stored in save data disk images, with each save data disk image corresponding to an individual save slot in the video game. Each save data disk image has its own file system and must be mounted before use and unmounted after use. The "mount" and "unmount" operations are performed by the respective APIs, which are considered control path APIs. Thus, an example of a data access flow on a video game console is: 1) (optionally, as needed) create a save data disk image; 2) mount the save data disk image; 3) open the file; 4) read from and / or write data to the file; 5) close the file; and 6) unmount the save data disk image.

[0023] A critical portion of the performance of a video game's data access flow resides within the data paths that utilize "read" and "write" APIs. For data "read" operations, the video game may expect the data to be returned within a certain amount of time to support the continued correct execution of the video game. Similarly, for data "write" operations, the video game may expect the data to be written within a certain amount of time to support the continued correct execution of the video game. If a storage solution is unable to "read" and / or "write" the data within the required time, bad things can happen during the execution of the video game. The exact types of bad things that can happen depend on the design of the video game and how the video game code is written. Some examples of bad things that can happen include video game crashes, undefined behavior within the video game, and / or audio stuttering within the video game, among others.

[0024] When a video game calls a data "read" API or a data "write" API, the video game passes one or more flags to the called API. These flags can be set to control how the called API executes. The behavior of both data reads and data writes within a video game depends on the flags the video game passes to the called API and the structure of the video game. For example, some video games can flag data read and data write operations as synchronous, meaning that the data "read" and data "write" APIs will prevent the video game from continuing to run until the data read and data write operations are completed. If a video game is written to call synchronous data reads or data writes from the code that performs game rendering, the video game will slow down or stutter whenever the read or write operation takes a long time to complete due to a slow storage device. However, if the video game flags the data write operation as asynchronous, as is typically done for data write operations, the video game will pass the data to be written to the operating system and continue running the video game without waiting for the data to be written. The location of data read and data write operations in a video game, and the flags associated with data read and data write operations, vary depending on the design of the video game. A well-written video game will at least attempt to perform data read and data write operations in smart areas of the video game code so as not to interfere with the smooth execution of the video game.

[0025] While control path APIs such as "mount," "open," "close," and "unmount" are important, they are often expected to disrupt the ongoing execution of a video game until they are completed. Therefore, timely execution of control path APIs is typically not critical to the performance of a video game. For example, a video game cannot perform any actions on data storage until the "mount" and "open" API calls are completed, and video game designers recognize this. Control path APIs are used in the systems and methods disclosed herein to achieve the goal of having a balanced storage solution that reduces costs by continuing to store data on performance-limited HDDs within cloud storage server 109, while ensuring that data currently needed by the video game is quickly available from storage devices within management servers 105-1 through 105-X.

[0026] In modern video games, a large amount of user data is generated. Depending on the video game and the video game platform (video game console), there may be various types of user data generated for a given user's video game. For example, in some embodiments, user data includes save data for the user, download data for the user, record data for the user (a record of the user's gameplay in a buffer or for the amount of time recently played), trophy data for the user, profile data for the user, video data for the user, audio data for the user, and pause and resume data (which captures the overall state of the online game system at a particular time), among other types of data for the user. Save data is part of the user data and is typically a data image of the video game that includes all data related to the state of the game and the user's state in the game at a particular save time. As a user plays a video game, there may be many save data images created for the user. For example, in some embodiments, save data for the user is generated at specific points during gameplay, such as when transitioning from one level to another or when changing from one scene to another, among many other times / examples when save data generation is triggered. Also, in some embodiments, save data generation is triggered by a menu selection by the user. A user's save data disk image is several megabytes (MB) in size.

[0027] In some embodiments, different types of user data may have different input / output (I / O) requirements, such as different types of data access performance requirements. Thus, in some embodiments, different types of user data may be stored in different cloud storage systems or different cloud storage servers that meet the different I / O requirements. Also, in some embodiments, the reliability of certain data is less important than the reliability of other data. Data reliability in this sense refers to ensuring the protection and availability of data. For example, the reliability of save data is very important to a user. However, the reliability of suspend and resume data, which can be very large, may not be as critical. Thus, losing suspend and resume data may not be as damaging to a user as losing a user's save data. Also, in some embodiments, some types of user data have a limited lifespan associated with them. For example, a system update or a video game update may render some types of user data no longer compatible or relevant to a user's video game play. Also, in some embodiments, different types of data may have different load time requirements. For example, system data and / or user profile data may need to be loaded from cloud storage server 109 to management servers 105-1-105-X as soon as a user logs into the system. Also, other data may be loaded from cloud storage server 109 to management servers 105-1-105-X when a particular game is started. Given the possible variations in I / O requirements, reliability, lifetime, and load timing among the various types of user and system data, there may be various performance tiers and various reliability tiers of cloud storage servers on which the user and system data are stored.

[0028] The control path API provides information about the data that a video game is attempting to access. For example, if a video game is attempting to read data from a specific file within a specific save data disk image, the video game must first mount the specific save data disk image using a "mount" API call and then open the specific file within the mounted save data disk image using an "open" API call. In this case, the "mount" and / or "open" API calls by the video game provide information that the video game is attempting to access data within a specific save data disk image. In some embodiments, information obtained from a control path API call made by a video game running on cloud game servers 103-1 through 103-N is used to trigger pre-fetching of data associated with the control path API call from slower HDD-based cloud storage server 109 and to store the fetched data in high-speed memory within management servers 105-1 through 105-X. Thereafter, any subsequent data path API calls made by the video game directed to the fetched data are completed quickly by accessing the data in high-speed memory within management servers 105-1 through 105-X. Then, when a video game is played using data in the high-speed memory in management servers 105-1-105-X, for example, an "unmount" control path API call causes the data in the high-speed memory in management servers 105-1-105-X to be flushed back to the slower, HDD-based cloud storage server 109. By "flushed," we mean that the data in the high-speed memory in management servers 105-1-105-X is copied to the slower, HDD-based cloud storage server 109 and then removed from the high-speed memory in management servers 105-1-105-X. In some implementations, the data may already have been written back to cloud storage server 109 by a "write" API call. This may occur via the operating system in a manner that is transparent to the running video game.The "unmount" API call always forces a flush of the data being unmounted to the cloud storage server 109. However, if data has already been written back to the cloud storage server 109 by a "write" API call as part of the normal course of operation, no data flush is required when the "unmount" API call is made, as the data has already been written back to the cloud storage server 109.

[0029] FIG. 2 shows a process diagram of a method for triggering data transfer from a slow data storage device in cloud storage server 109 to high-speed data storage 119 in management server 105-1 using a control path API call by cloud game server 103-1 prior to a data path API call by cloud game server 103-1 to access the transferred data, according to some embodiments. In a first step, cloud game server 103-1 issues a control path API call to mount a specific save data disk image (SaveData2 in the example of FIG. 2). In response, management server 105-1 mounts the specific save disk image. In a second step, the specific save disk image (e.g., SaveData2) that cloud game server 103-1-103-N requested to be mounted is copied from cloud storage server 109 to high-speed data storage 119 in management server 105-1. In some embodiments, high-speed data storage 119 is a computer memory device such as RAM or storage-class memory, or another type of computer memory substantially equivalent to RAM and / or storage-class memory. In some embodiments, high-speed data storage 119 is NVMe-level storage, such as an SSD. In a third step, cloud gaming server 103-1 issues one or more data path API calls to read and / or write data to the particular save disk image. However, rather than having cloud storage server 109 provide the data path API calls from cloud gaming server 103-1, the data path API calls are instead provided by management server 105-1, which currently has the particular save disk image in its high-speed data storage 119.

[0030] In a fourth step, the cloud gaming server 103-1 issues a control path API call to unmount the particular save data disk image. In response to the unmount API call, the storage system performs a fifth step in which the management server 105-1 operates to copy the data of the particular save disk image currently residing in high-speed data storage 119 back to the cloud storage server 109. In some embodiments, if data written to the particular save disk image in high-speed data storage 119 is copied back to the cloud storage server 109 during the normal course of operation (e.g., when written to high-speed data storage 119), the data that needs to be copied back to the cloud storage server 109 may not be present at the time the cloud gaming server 103-1 issues the unmount API call. It should be understood that in response to the unmount API call by the cloud gaming server 103-1, the management server 105-1 operates to ensure that the current data of the particular save disk image is stored in the cloud storage server 109 just as it was stored in the high-speed data storage 119 of the management server 105-1 at the time the unmount API call was issued. Once the management server 105-1 has verified that all of the current data for a particular save disk image is properly stored within the cloud storage server 109, the management server 105-1 operates to unmount the particular save disk image in accordance with the unmount API call.

[0031] In some embodiments, while a user is playing a video game, the user's game data is maintained on the management servers 105-1-105-X. The management servers 105-1-105-X should have sufficient storage capacity to handle the storage needs of the user while he or she is playing the video game. In some embodiments, the management servers 105-1-105-X operate to set aside a certain amount of storage for each active user. In the event of storage pressure / overload on the management servers 105-1-105-X, one option is for the management servers 105-1-105-X to direct storage access API calls to be handled by the cloud storage server 109 rather than by the management servers 105-1-105-X. Alternatively, the management servers 105-1-105-X can flash back some data to the cloud storage server 109 to relieve the storage pressure / overload on the management servers 105-1-105-X. In this case, management servers 105-1-105-X flash back either all data or a portion of the data to cloud storage server 109. In some embodiments, the act of flashing data from management servers 105-1-105-X back to cloud storage server 109 to relieve storage pressure / overload on management servers 105-1-105-X is based on an analysis of the current storage needs and usage patterns of all users currently storing data on management servers 105-1-105-X. In some embodiments, when management servers 105-1-105-X flash back data to cloud storage server 109 to relieve storage pressure / overload, management servers 105-1-105-X copies back some of the data from cloud storage server 109 in anticipation of it being needed by one or more video games running on one or more cloud game servers 103-1-103-N served by management servers 105-1-105-X.In some embodiments, when a video game writes data to management servers 105-1-105-X, that data remains on management servers 105-1-105-X for access by the video game, even if the data is copied to cloud storage server 109 in the future. In some embodiments, there are various algorithms implemented to sort the data stored on management servers 105-1-105-X at a given time in order to prioritize flashing the data back to cloud storage server 109 if there is pressure / overload on management servers 105-1-105-X. Also, in some embodiments, in the event of storage pressure / overload on management servers 105-1-105-X, management servers 105-1-105-X can alternatively notify cloud game servers 103-1-103-N and / or video games of the storage pressure / overload situation, causing cloud game servers 103-1-103-N and / or video games to trigger a data flashback to cloud storage server 109 to alleviate the storage pressure / overload situation. This option can be used to prevent data loss by preventing cloud game servers 103-1-103-N and / or video games from performing any data updates when management servers 105-1-105-X act to flash data back to cloud storage server 109.

[0032] In some embodiments, the process of FIG. 2 may be particularly useful when a user initiates an online game session. When a user starts an online video game, cloud game servers 103-1-103-N must load the executable code for the online video game from storage server 107. Cloud game servers 103-1-103-N may also need to load other data, such as audio data and / or texture data, among other things, to start the video game. It takes approximately 5 seconds for cloud game servers 103-1-103-N to load this data and run the video game to a game menu display state. This time can be used to load the user's user data from cloud storage server 109 to management servers 105-1-105-X. For example, this time can be used to load the user's user data corresponding to the user's last save point in the video game from cloud storage server 109 to management servers 105-1-105-X when the video game is launched. Next, when the video game presents game menu options such as New Game or Load Game, the user data for the user's last save point in the video game is already stored in high-speed data storage 119 of management server 105-1-105-N and is ready for immediate and fast access by cloud game server 103-1-103-N if the user selects to continue playing the video game from the last save point. Next, when the user selects to load a previously saved game, an open API call causes the video game running on cloud game server 103-1-103-N to open the user's save data file in the previous save data disk image. This open API call is processed by management server 105-1-105-X, and the requested save data file is read from high-speed data storage 119 within management server 105-1-105-X rather than from the slower cloud storage server 109.It should be appreciated that by reading the user's save data file from high-speed data storage 119 in management servers 105-1-105-X, the user can begin playing the video game more quickly. During execution of the video game on cloud game servers 103-1-103-N, the video game can issue additional data storage access API calls that trigger the copying of additional data from cloud storage server 109 to high-speed data storage 119 in management servers 105-1-105-X, thereby enabling faster access to data requested by the video game from high-speed data storage 119 in management servers 105-1-105-X.

[0033] In some embodiments, cloud gaming servers 103-1-103-N maintain a database of where user data is stored on cloud storage servers 109. In some embodiments, management servers 105-1-105-X shield cloud gaming servers 103-1-103-N from cloud storage servers 109, so that cloud gaming systems 103-1-103-N do not need to keep track of where data is stored on cloud storage servers 109. In some embodiments, management servers 105-1-105-X maintain a storage tracking database that contains information indicating where various data is stored within cloud storage servers 109. When management server 105-1-105-X receives a request for a particular type of user data from cloud gaming server 103-1-103-N, via a data storage access API call, management server 105-1-105-X operates to query the storage tracking database to determine the identity of cloud storage server 109 on which the requested data is located and where the requested data is located on the identified cloud storage server 109. Management server 105-1-105-X then transmits the request for the data to the identified cloud storage server 109. In some embodiments, the storage tracking database is stored on cloud storage server 109 and loaded into high-speed data storage 119 of management server 105-1-105-X where the storage tracking database is accessed and queried.

[0034] In some embodiments, the storage tracking database on management servers 105-1-105-X is queried to determine which user data files need to be updated at other data centers to maintain current redundancy and synchronization of user data at multiple data centers when a user is traveling, and / or for data backup purposes, and / or other reasons. Also, in some embodiments, a user plays video games on a local gaming console 127 rather than on a cloud gaming server 103-1-103-N. In these embodiments, the storage tracking database on management systems 105-1-105-X is used when local gaming console 127 needs to access data stored on cloud storage server 109 via management servers 105-1-105-N. Also, in some embodiments, the storage tracking database on management servers 105-1-105-X is used to enable synchronization of a user's local play of a video game on local gaming console 127 with a user's cloud game play of a video game on cloud gaming servers 103-1-103-N.

[0035] In some embodiments, the local game console 127 includes an SSD that wears out as data is written to it. Knowing this, video game developers program their video games to direct storage operations with a limited SSD lifespan, which corresponds to the limited number of program / write cycles the memory units in the SSD can withstand. Video game developers also program their video games with a consideration that writing small amounts of data to the SSD is particularly problematic. Thus, in many video games, the flash memory of an SSD-equipped local game console 127 is not permitted to write small blocks of data to the SSD, but instead must retain the data until enough data has accumulated to write larger blocks of data to the SSD. To reduce the number of write / program cycles to the SSD, video games are programmed to use "transactions" for storage access. Using transactions for storage access also has the advantage of, among other things, reducing the risk of data corruption. The idea behind using transactions for storage access is that the video game continues to use POSIX API calls such as read and write, but the written data is stored only in RAM until the transaction is complete, when the video game calls the "commit" API to trigger the flushing of data stored in RAM to the SSD. The unmount API is an "implicit commit." Thus, the data access flow in a local game console 127 equipped with an SSD that uses transactions for storage access is: 1) (optionally, if needed) create a save data disk image; 2) create a transactional resource; 3) mount the save data disk image; 4) open a file; 5) read and / or write data to the file (writes to RAM only); 6) close the file; 7) commit the data; 8) delete the transactional resource; 9) unmount the save data disk image.Considering that the cloud gaming system is capable of running the same video games that run on a local gaming console 127 equipped with an SSD, cloud gaming servers 103-1 through 103-N, management servers 105-1 through 105-X, and cloud storage server 109 are collectively configured to handle transactional usage for storage access.

[0036] In some embodiments, the transaction data buffer can be provided by either the video game or the cloud gaming systems 103-1-103-N. In some embodiments, the transaction data buffer is in RAM. In some embodiments, the cloud gaming systems 103-1-103-N let the video game provide the transaction buffer because the size of the transaction buffer is small. Also, by having the video game provide the transaction buffer, the video game is aware of the limited capacity of the transaction buffer. In some embodiments, if the video game writes too much data using the standard data path "write API," the write operation fails with an error that there is not enough space to complete the write operation.

[0037] In some embodiments, when cloud game server 103-1-103-N calls the commit API, cloud game server 103-1-103-N also enables management server 105-1-105-X to commit its buffer changes back to cloud storage server 109. In some embodiments, if the commit API call is a blocking API call (by default or by a flag setting), the commit API call causes the video game to wait until management server 105-1-105-X has completed flashing back the data to cloud storage server 109. Otherwise, data corruption could occur if management server 105-1-105-X crashes.

[0038] In some embodiments, to prevent data corruption, in case a power outage or other error condition occurs during a commit / unmount operation, there is a mechanism to revert to the data state that existed before performing the commit / unmount operation. For example, in some embodiments, before committing data, the system reads any of the data that is expected to be updated so that the read data can be recovered if a problem occurs with the commit operation. In some embodiments, management servers 105-1 through 105-X are programmed to read any of the data that is expected to be updated before performing a commit API operation. Management servers 105-1 through 105-X are programmed to track what data changes are requested and how the data is requested to be changed. Additionally, in some embodiments, the cloud storage server 109 is configured to support transactions. In these embodiments, the cloud storage server 109 tracks data changes upon mount by the management servers 105-1-105-X. A commit / unmount API call then triggers a flush of the data changes. If the flush of data changes from the management servers 105-1-105-X to the cloud storage server 109 fails, the cloud storage server 109 can be used to revert the data changes.

[0039] 3 shows an exemplary diagram of some internal components of cloud gaming server 103-1 and management server 105-1, according to some embodiments. Root complex 311 of cloud gaming server 103-1 has peripheral component interconnect express (PCIe) port 313 connected to PCIe port 315 of PCIe switch 305 via PCIe link 309. Root complex 317 of management server 105-1 also has PCIe port 316 connected to PCIe port 319 of PCIe switch 305 via PCIe link 307. PCIe switch 305 includes logic for bridging the memory address spaces of management server 105-1 and cloud gaming server 103-1. For example, the logic for bridging the memory address spaces of management server 105-1 and cloud gaming server 103-1 may include a non-transparent bridge (NTB) or a specialized direct memory access (DMA) engine that communicates with both management server 105-1 and cloud gaming server 103-1 and handles the necessary memory address translation. In some embodiments, the DMA engine may be implemented within PCIe switch 305. It should be understood that both root complex 311 of cloud gaming server 103-1 and root complex 317 of management server 105-1 include a PCIe interface or an equivalent interface, such as, for example, a Gen-Z interface defined in accordance with the Gen-Z Communications / Interconnect Specification developed by the Gen-Z Consortium or other future-developed interface that is essentially equivalent to either a PCIe interface or a Gen-Z interface.

[0040] For example, in some embodiments, PCIe switch 305 optionally includes NTB 321. When PCIe switch 305 includes NTB 321 and is in NTB mode, NTB 321 functions to translate physical memory addresses in memory access commands written by cloud gaming server 103-1 to data I / O (input / output) command buffers into memory addresses retrievable by management server 105-1 within computer memory 323 of management server 105-1. NTB 321 functionally operates to modify memory addresses at the PCIe transaction layer, where transaction layer packets (TLPs) are assembled and disassembled. In some embodiments, NTB 321 also operates to modify requester identifiers. When cloud game server 103-1 generates a command buffer, such as through storage CPU 331, the command buffer is ultimately wrapped into a TLP by root complex 311. Storage CPU 331 operates to ensure the accuracy of any memory addresses in the command buffer generated by storage CPU 331. Storage CPU 331 also operates to ensure that the memory addresses in the command buffer are translated addresses when management server 105-1 needs to write data back to cloud game server 103-1. NTB 321 comes into play when cloud game server 103-1 writes the command buffer to a memory address within management server 105-1. In this case, the memory address to which the command buffer is written is translated by NTB 321 and / or by output memory management unit (IOMMU) 381 in computer memory 323 of management server 105-1. IOMMU 381 maps virtual memory addresses seen by the device to physical memory addresses. Also, in this case, the contents of the command buffer itself are not modified by NTB 321 or IOMMU 381.

[0041] NTB mode allows multiple root complexes to communicate with each other, i.e., exchange data with each other. In some embodiments, if PCIe switch 305 is connected to a secondary PCIe switch that has only NTB devices connected to it, the secondary PCIe switch only forwards transaction layer packets (TLPs) and therefore does not need to use NTB. Computer memory 323 represents high-speed data storage 119 mentioned with respect to FIG. 2. In various embodiments, computer memory 323 can be RAM or storage-class memory, or another type of computer memory substantially equivalent to RAM and / or storage-class memory. NTB 321 handles the translation of memory addresses in data I / O command buffers and provides the translated memory addresses to management server 105-1. If the PCIe switch 305 does not use the NTB 321, the memory access commands written to the data I / O command buffer may be generated to include other metadata, such as a host identifier or other types of metadata, that can be used to determine the memory address in the computer memory 323 of the management server 105-1 to which the memory address in the data I / O command buffer needs to be mapped. In some embodiments in which the PCIe switch 305 does not use the NTB 321, a DMA engine can be used to send the packets across the PCIe switch 305 to the correct destination. Also, in some embodiments, if the PCIe switch 305 does not use the NTB 321 and is connected to a secondary PCIe switch, the secondary PCIe switch operates to forward the packets to the correct destination.

[0042] Cloud gaming server 103-1 includes at least one CPU 325 connected to an internal data bus 327. In various embodiments, cloud gaming server 103-1 may also include one or more GPUs 329 connected to internal data bus 327, one or more storage CPUs 331 connected to internal data bus 327, one or more decryption / encryption engines 333 connected to internal data bus 327, and / or one or more decompression / compression engines 335 connected to internal data bus 327. Internal data bus 327 is connected to root complex 311 of cloud gaming server 103-1. In various embodiments, GPU 329 is a processor configured to perform parallel operations to render images, animations, and videos for display on an electronic display screen. Additionally, in some embodiments, the parallel processing capabilities of GPU 329 can be used for vector processing in non-graphics applications that require repetitive calculations.

[0043] In various embodiments, storage CPU 331 is configured to handle storage command requests and allocate storage on a particular type of storage media. In various embodiments, storage CPU 331 and main CPU 325 can have the same or different architectures. For example, in some embodiments, storage CPU 331 can be an Advanced RISC (Reduced Instruction Set Computer) Machine (ARM) processor or a MIPS processor, where main CPU 325 is an x86 processor. It should be understood that in various embodiments, storage CPU 331 can be essentially any type of suitable computer processor, and main CPU 325 can be essentially any type of suitable computer processor. Also, in some embodiments, instead of having both main CPU 325 and storage CPU 331, cloud gaming servers 103-1-103-N only have main CPU 325. In this case, operations that would be performed by storage CPU 331 if both CPUs were present are performed by main CPU 325. However, for purposes of discussion herein, cloud gaming servers 103-1-103-N are considered to include both main CPU 325 and storage CPU 331. The particular type of storage media to which storage CPU 331 allocates data for storage may vary depending on the performance capabilities, e.g., data read and / or data write speeds, of the storage media and / or the form in which the data is stored on the storage media, such as file-based data storage, folder-based data storage, or byte-based data storage. In some embodiments, storage CPU 331 operates to divide data into multiple data chunks for placement in respective locations within computer memory 337 of cloud gaming server 103-1, with various storage locations within computer memory 337 being tracked and updated based on a storage table.In various embodiments, computer memory 337 may be RAM or storage-class memory, or another type of computer memory substantially equivalent to RAM and / or storage-class memory. Storage CPU 331 may also function to optimize storage based on input / output per second (IOPS). In some embodiments, IOPS-based storage optimization may be correlated to the access speed of available storage media, which may include rotating storage media, solid-state storage media, and / or hybrid storage media. In various embodiments, decryption / encryption engine 333 is configured to apply cryptographic algorithms to decrypt encrypted data and encrypt unencrypted data, and is a dedicated processor for applying cryptographic algorithms. In various embodiments, decompression / compression engine 335 is configured to apply algorithms to decompress compressed data and compress uncompressed data, and is configured to be dedicated to those algorithms.

[0044] In some embodiments, computer memory 337 of cloud gaming server 103-1 is connected to root complex 311. In some embodiments, cloud gaming server 103-1 also includes a DMA controller 339 connected to root complex 311. In some embodiments, computer memory 337 and / or DMA controller 339 can be connected to internal data bus 327, as indicated by lines 361 and 363, respectively. Also, in some embodiments, cloud gaming server 103-1 can include one or more NVMe SSDs 341 connected to PCIe ports 345 of root complex 311 via PCIe link 343. Also, in some embodiments, cloud gaming server 103-1 can include one or more SATA controllers 347 connected to PCIe ports 351 of root complex 311 via PCIe link 349. SATA controller 347 can act as a bridge between HDD 353 and one or more of the PCIe fabrics. Also, in various embodiments, one or more other PCIe-enabled component device(s) 355 may be connected to respective PCIe port(s) 357 of root complex 311 via respective PCIe links 359. In various embodiments, other PCIe-enabled component device(s) 355 may include one or more GPUs, one or more field programmable gate arrays (FPGAs), one or more network adapters, one or more SSDs, one or more SATA / HHDs, among other devices. It should be understood that the architecture of cloud gaming server 103-1 shown in FIG. 3 is provided by way of example. In various embodiments, cloud gaming server 103-1 may include more or fewer components than those shown in the example of FIG. 3.

[0045] Management server 105-1 includes at least one CPU 365 connected to an internal data bus 367. In various embodiments, management server 105-1 may also include one or more GPUs 369 connected to internal data bus 367 and / or one or more network interface cards (NICs) 373 connected to internal data bus 367. Internal data bus 367 is connected to a root complex 317 of management server 105-1. Management server 105-1 may also include one or more NVMe SSDs 371 connected to root complex 317. Also, in some embodiments, GPU 369 and / or NIC 373, and / or other devices may be configured to use PCIe and connect directly to root complex 317 rather than to internal data bus 367.

[0046] The systems and methods disclosed herein utilize a PCIe fabric connecting cloud gaming server 103-1 to cloud management server 105-1 to enable CPU 325 and / or storage CPU 331 and / or DMA controller 339 (and / or other devices) on cloud gaming server 103-1 to send messages and copy data between computer memory 323 of management server 105-1 and computer memory 337 of cloud gaming server 103-1, and vice versa. The result of connecting cloud gaming server 103-1 and management server 105-1 using a PCIe fabric, and after configuring PCIe switch 305, is a “shared memory” mechanism across cloud gaming server 103-1 and management server 105-1. Cloud gaming server 103-1 can directly access specific portions 377 of computer memory 323 within management server 105-1 with very little overhead, e.g., bypassing CPU 365 of management server 105-1, as indicated by arrow 379. Direct access of computer memory 323 in server management server 105-1 by cloud gaming server 103-1 may be directed by any device in cloud gaming server 103-1 that can manage data storage requests and that is connected to PCIe switch 305 via root complex 311, such as CPU 325 and / or storage CPU 331 and / or DMA controller 339. However, it should be understood that this may not be the most efficient use of CPU 325 of cloud gaming server 103-1 to manage data storage requests. Thus, storage CPU 331 and / or DMA controller 339 may be used primarily to manage data storage requests of cloud gaming server 103-1 by generating memory access commands that are written to data I / O command buffers in computer memory 323 of management server 105-1.

[0047] In addition to providing a "shared memory" mechanism, the connection of cloud game server 103-1 to management server 105-1 via PCIe switch 305 provides for the transmission of "messages" from cloud game server 103-1 to management server 105-1, and vice versa, by writing messages directly to computer memory 337 of cloud game server 103-1, or to computer memory 323 of management server 105-1, or to some memory within PCIe switch 305. The transmission of messages between cloud game server 103-1 and management server 105-1 can be used to facilitate the execution of memory access commands written to data I / O command buffers. For example, if cloud game server 103-1 expects management server 105-1 to execute some memory access command that will result in some requested data being retrieved from a data storage device and stored somewhere in shared memory existing between cloud game server 103-1 and management server 105-1, a message can be written by management server 105-1 to a memory location monitored by cloud game server 103-1 to notify cloud game server 103-1 of when and where the requested data will be available to management server 105-1.

[0048] In some embodiments, a message may be sent by either cloud game server 103-1 or management server 105-1 to one or more doorbell register(s) 375 exposed via PCIe switch 305. In some embodiments, a write to doorbell register 375 by cloud game server 103-1 generates an interrupt on management server 105-1, which causes management server 105-1 to process the interrupt. In some embodiments, processing of the interrupt by management server 105-1 causes management server 105-1 to read the message (some data) from a specific location in shared memory that exists between cloud game server 103-1 and management server 105-1. Similarly, a write to doorbell register 375 by management server 105-1 generates an interrupt on cloud game server 103-1, which causes cloud game server 103-1 to process the interrupt. In some embodiments, processing of the interrupt by cloud game server 103-1 causes cloud game server 103-1 to read a message (some data) from a specific location in a shared memory that exists between cloud game server 103-1 and management server 105-1.

[0049] The content of the message varies depending on the use case. In some use cases, the message may convey a request to read a certain amount of data from a storage device and store the read data in shared memory existing between the cloud game server 103-1 and the management server 105-1. Then, after the request in the message is completed, another message may be sent to notify completion. For example, if the cloud game server 103-1 sends the management server 105-1 a first message requesting that specific data be read from data storage, the management server 105-1 executes the request in the first message, and a second message is sent to the cloud game server 103-1 notifying the cloud game server 103-1 of the completion of the first message request, and the specific requested data is stored in shared memory existing between the cloud game server 103-1 and the management server 105-1.

[0050] Cloud gaming server 103-1 and management server 105-1 are independent systems, each with its own memory map. The memory map holds the memory addresses of computer memory and other devices within the system. PCIe switch 305 occupies a user-configurable region of memory address space within each memory map of cloud gaming server 103-1 and management server 105-1. In various embodiments, depending on the configuration of PCIe switch 305, physical memory addresses in computer memory 337 of cloud gaming server 103-1 can be mapped to the same or different physical memory addresses in computer memory 323 of management server 105-1.

[0051] In some embodiments, for security and / or other reasons, cloud management server 105-1 can be configured to hide its real physical memory addresses, such as by implementing IOMMU 381, which maps device-visible virtual memory addresses to physical memory addresses. IOMMU 381 is configured to map device memory addresses to physical memory addresses, and a conventional memory management unit (MMU) is configured to map virtual memory addresses to physical memory addresses. With a conventional MMU, the virtual memory addresses are contiguous so that an application can see and manipulate a block of memory, such as a 64 MB block of memory, but in reality, the MMU maps the contiguous virtual memory addresses to multiple disjoint physical memory addresses, hiding the memory mapping from the application. Similar to a typical MMU, IOMMU 381 maps virtual memory addresses visible to cloud gaming server 103-1 to multiple disjointed physical memory addresses within computer memory 323, hiding the memory mapping from management server 105-1. In this manner, using IOMMU 381, cloud gaming server 103-1 can view a contiguous block of virtual memory addresses that exist on management server 105-1 while hiding the complexity of how that contiguous block of virtual memory addresses actually maps to different physical memory addresses across computer memory 323 of management server 105-1. In some embodiments, IOMMU 381 is included in root complex 317 of management server 105-1. In some embodiments, root complex 311 of cloud gaming server 103-1 can also include IOMMU 383.

[0052] In the configuration of FIG. 3 , IOMMU 381 may allow cloud game server 103-1 to view a particular memory address (device address) as having a one-to-one correspondence with a particular physical memory address within computer memory 323 of management server 105-1. However, in reality, IOMMU 381 may remap a particular memory address as seen by cloud game server 103-1 to a different physical memory address within computer memory 323 of management server 105-1. Also, in some embodiments, IOMMU 381 may be used to block access to one or more specified physical memory regions within computer memory 323 of management server 105-1 that are not authorized for access by cloud game server 103-1. Also, because IOMMU 381 effectively hides memory fragmentation within computer memory 323 of management server 105-1 from cloud game server 103-1, use of IOMMU 381 may simplify DMA transfer operations within cloud game server 103-1. For example, if IOMMU 381 were not used to present fragmented physical memory addresses in computer memory 323 to cloud game server 103-1 as a contiguous virtual block of memory addresses, cloud game server 103-1 would need to perform multiple DMA transfer operations or scatter / gather transactions to read data across the fragmented physical memory addresses in computer memory 323. However, by using IOMMU 381 to present the fragmented physical memory addresses in computer memory 323 to cloud game server 103-1 as a contiguous virtual block of memory addresses, cloud game server 103-1 can perform a single DMA transfer operation to read data across the fragmented physical memory addresses in computer memory 323.

[0053] In some embodiments, management server 105-1 is connected to cloud storage system 390 via NIC 373, as indicated by connection 392. Cloud storage system 390 includes one or more cloud storage servers 109-1 through 109-Y. Each cloud storage server 109-1 through 109-Y includes a respective NIC 387-1 through 387-Y. Each cloud storage server 109-1 through 109-Y also includes one or more HDDs 389-1 through 389-Y. In some embodiments, some of cloud storage servers 109-1 through 109-Y also include one or more NVMe SSDs or other types of data storage devices. In some embodiments, cloud storage system 390 is configured and operated to distribute data storage among multiple physical storage media (HDDs) and to optimize use of the multiple physical storage media using data deduplication methods. The multiple physical storage media accessible / controlled by cloud storage system 390 can be located in multiple different storage boxes interconnected by high-speed interconnects and high-speed switches. In some embodiments, the multiple different storage boxes containing the multiple physical storage media accessible / controlled by cloud storage system 390 can be located at different locations within a data center or across multiple geographically distributed data centers. In some embodiments, a caching system can be used to manage data storage across multiple data centers so that data used or needed for a particular requesting entity, such as a particular remote client device used by a user to play a cloud gaming application, can be moved to a particular data center that is closest to the requesting entity or that provides an acceptable quality of service to the requesting entity. In some embodiments, the caching system can utilize a defined migration algorithm to migrate user data and / or game data to a particular data center.

[0054] Typically, access to game data is read-only. However, access to save data includes both read and write operations. Access to user data may be managed by management servers 105-1-105-X. Management servers 105-1-105-X mount requested save data disk images from cloud storage servers 109-1-109-Y and implement the protocols necessary for data communication with cloud storage servers 109-1-109-Y. In various embodiments, TCP-based data communication protocols, such as Ceph, NFS, Amazon S3 (HTTP-based), or some other type of object, file, or block storage protocol, may be used for various data communications between management servers 105-1-105-X and cloud storage servers 109-1-109-Y. As mentioned above, in some embodiments, cloud game servers 103-1-103-N communicate data with management servers 105-1-105-X using PCI Express. In some embodiments, cloud game servers 103-1-103-N communicate data with management servers 105-1-105-X over a network via NIC 393 connected to NIC 373, as indicated by connection 394. Data communication between cloud game servers 103-1-103-N and management servers 105-1-105-X occurs according to one or more of a variety of data communication protocols, such as Ethernet (TCP / IP), RDMA, InfiniBand, among others.

[0055] The data communication path taken by save data (and other user data) is somewhat similar to that of game data. In both cases, the CPU 325 of each cloud game server 103-1-103-N generates command buffers corresponding to different control path APIs and data path APIs. The command buffers are processed by the storage CPU 331 of each cloud game server 103-1-103-N. The storage CPU 331 instructs the management servers 105-1-105-X to transmit the command buffers, which have cloud storage mounted from one or more of the cloud storage servers 109-1-109-Y. In some embodiments, the control path APIs can be transmitted over a network connection such as Ethernet, RDMA, or InfiniBand, among others. However, performance-critical command buffers, such as those containing data path APIs, are transmitted to the management servers 105-1-105-X over a high-speed data connection, such as a PCIe fabric. Management servers 105-1-105-X are programmed to operate CPU 365 to receive and process command buffers received from cloud game servers 103-1-103-N. According to the command buffers received from cloud game servers 103-1-103-N, management servers 105-1-105-X operate to retrieve data from cloud storage system 390 as needed using network protocols such as Ethernet, Ceph, NFS, and S3, among others. For example, in some embodiments, a "mountSaveDiskImage" control path API triggers the transfer of data from HDD 389-1-389-Y in a given cloud storage server 109-1-109-Y to RAM 323 or NVMe SSD 371 in management server 105-1-105-X. In some embodiments, read / write data path APIs access or modify data in RAM 323 of management servers 105-1-105-X. The process by which data is written back to cloud storage system 390 depends on whether transactions are used. Without transactions, the write / close / unmount APIs flush back data from management servers 105-1 to 105-X to cloud storage system 390. With transactions, the commit API flushes back any changes to data from management servers 105-1 to 105-X to cloud storage system 390.

[0056] Various embodiments of cloud gaming server 100 are disclosed herein to include cloud gaming server 103 (cloud gaming server 103 is any of cloud gaming servers 103-1 through 103-N), cloud storage server 109 (cloud storage server 109 is any of cloud storage servers 109-1 through 109-Y), and management server 105 (management server 105 is any of management servers 105-1 through 105-X). Cloud gaming server 103 is configured to execute a video game according to input received from controller device 125 of user 121. Cloud gaming server 103 is configured to direct the transmission of a video stream reflecting the play of the video game by user 121 to local computing system 120 of user 121. Cloud storage server 109 is in data communication with cloud gaming server 103. The cloud storage server 109 includes a data storage device 389 for storing user data of the user 121 (the data storage device 389 is a data storage device 389-1 to 389-Y in the corresponding cloud storage server 109-1 to 109-Y). The user data includes data recording the user's 121 play of the video game. The management server 105 is in data communication with both the cloud gaming server 103 and the cloud storage server 109. The management server 105 includes a high-speed data storage device, such as an NVMe SSD 371, that operates at a faster data speed and lower latency than the data storage device in the cloud storage server 109. In some embodiments, the data storage device in the cloud storage server 109 is an HDD, and the high-speed storage device is an NVMe SSD. The management server 105 is programmed to receive a data access request issued by a video game running on the cloud gaming server 103. The data access request identifies the requested data stored in the data storage device 389 in the cloud storage server 109. The management server 105 is programmed to copy the requested data from the data storage device 389 in the cloud storage server 109 to the high-speed data storage device 371 in the management server 105. The management server 105 is also programmed to use the high-speed data storage device 371 to respond to data access requests instead of the cloud storage server 109 responding to the data access requests.

[0057] It should be understood that there is a difference in storage performance between the management servers 105-1 to 105-X and the cloud storage server 109, with the NVMe SSDs of the management servers 105-1 to 105-X operating to provide data to the cloud game servers 103-1 to 103-N at much higher speeds than the HDDs of the cloud storage server 109. There is also a difference in network performance between the management servers 105-1 to 105-X and the cloud storage server 109, with the management servers 105-1 to 105-X providing faster network data transmission speeds (higher network bandwidth) between the management servers 105-1 to 105-X and the cloud storage server 109 compared to the cloud storage server 109. Utilizing the management servers 105-1 to 105-X as a data cache can limit the data load in network communications with the cloud storage server 109. Using the management servers 105-1 to 105-X as a data cache ensures that read operations from the cloud storage server 109 can be performed at very high speeds. In some embodiments, without having the management servers 105-1 to 105-X provide a cache, the effective data communication bandwidth with the cloud storage server 109 may be significantly reduced.

[0058] In some embodiments, cloud storage server 109 is located at a different location than cloud gaming servers 103-1-103-N. In some embodiments, many cloud gaming servers 103-1-103-N are located at an Internet Service Provider (ISP). This is inexpensive for operational reasons because the ISP provides power and Internet connectivity for free. In such embodiments, the network bandwidth between the ISP's data center and the cloud gaming data center should be good, but not necessarily sufficient for cloud gaming servers 103-1-103-N to write data directly to the NVMe SSD drives in cloud storage server 109. Therefore, in these embodiments, even if cloud storage server 109 implements NVMe SSD drives, which are faster than HDDs, the data caching functionality provided by management servers 105-1-105-X is still beneficial.

[0059] In some embodiments, the management server 105 is programmed to communicate bidirectionally with the cloud storage server 109 using a first network storage protocol, and the management server 105 is programmed to communicate bidirectionally with the cloud gaming server 103 using a second network storage protocol that is different from the first network storage protocol. In some embodiments, the second network storage protocol is PCIe.

[0060] In some embodiments, the data access request issued by the video game running on the cloud gaming server 103 is a request to mount a specific group of data. The management server 105 is programmed to respond to the subsequent data access request to the specific group of data issued by the video game running on the cloud gaming server 103 by using the high-speed storage device 371 instead of having the cloud storage server 109 respond to the subsequent data access request. In some embodiments, the specific group of data is a data object, a data file, or a data block. In some embodiments, the specific group of data is a save data disk image for the video game that contains save data for a user's play of the video game.

[0061] In some embodiments, a subsequent data access request issued by a video game running on the cloud gaming server 103 is either a read API call, a write API call, an unmount API call, or a commit API call. In some embodiments, the management server 105 is programmed to respond to the write API call by writing data to a specific group of data in the high-speed storage device 371 within the management server 105. In some embodiments, the management server 105 is programmed to respond to the write API call by writing data to both a specific group of data in the high-speed storage device 371 within the management server 105 and a specific group of data in the data storage device 389 within the cloud storage server 109. In some embodiments, the management server 105 is programmed to respond to the read API call by reading data into a specific group of data in the high-speed storage device 371 within the management server 105. In some embodiments, the management server 105 is programmed to respond to the unmount and commit API calls by ensuring that all data currently within a particular classification of data in the high-speed storage device 371 within the management server 105 is also currently within a particular classification of data in the data storage device 389 within the cloud storage server 109.

[0062] 4 shows a flowchart of a method for operating the cloud gaming system 100, according to some embodiments. The method includes an operation 401 for executing a video game on the cloud gaming server 103 according to input received from the controller device 125 of the user 121. The method also includes an operation 403 for operating the cloud gaming server 103 to direct the transmission of a video stream reflecting the user 121's play of the video game to the user's 121's local computing system 120. The method also includes an operation 405 for causing the cloud storage server 109 to store user data of the user 121. The user data includes data recording the user 121's play of the video game. The method also includes an operation 407 for receiving a data access request at the management server 105. The data access request is issued by a video game running on the cloud gaming server 103. The data access request identifies requested data stored on a data storage device 389 in the cloud storage server 109. The method also includes an operation 409 for copying the requested data from the data storage device 389 in the cloud storage server 109 to a high-speed data storage device 371 in the management server 105. The high-speed data storage device 371 operates at a higher data rate and lower latency than the data storage device 389 in the cloud storage server 109. In some embodiments, the data storage device 389 in the cloud storage server 109 is an HDD, and the high-speed storage device 371 in the management server 105 is an NVMe SSD. The method further includes an operation 411 for responding to the data access request using the high-speed data storage device 371 in the management server 105 instead of the cloud storage server 109 responding to the data access request.

[0063] In some embodiments, the method includes using a first network storage protocol for bidirectional data communication between the management server 105 and the cloud storage server 109, and using a second network storage protocol for bidirectional data communication between the management server 105 and the cloud gaming server 103, wherein the second network storage protocol is different from the first network storage protocol. In some embodiments, the second network storage protocol is PCIe.

[0064] In some embodiments, the data access request is a request to mount a particular classification of data. In some embodiments, the particular group of data is a data object, a data file, or a data block. In some embodiments, the particular group of data is a save data disk image for a video game, the save data disk image including save data for a user's play of the video game. The method also includes receiving a subsequent data access request at the management server 105, the subsequent data access request issued by a video game running on the cloud gaming server 103, the subsequent data access request being directed to the particular classification of data. The method also includes responding to the subsequent data access request using a high-speed data storage device 371 within the management server 105, instead of the cloud storage server 109 responding to the subsequent data access request.

[0065] In some embodiments, the subsequent data access request is either a read API call, a write API call, an unmount API call, or a commit API call. In some embodiments, the method includes operating the management server 105 to respond to the write API call by writing the data to a particular classification of data in a high-speed storage device 371 in the management server 105. In some embodiments, the method includes operating the management server 105 to respond to the write API call by writing the data to both a particular classification of data in the high-speed storage device 371 in the management server 105 and a particular classification of data in a data storage device 389 in the cloud storage server 109. In some embodiments, the method includes operating the management server 105 to respond to the read API call by reading data from the particular classification of data in the high-speed storage device 371 in the management server 105. In some embodiments, the method includes operating the management server 105 to respond to the unmount and commit API call by ensuring that all data currently in the particular classification of data in the high-speed storage device 371 in the management server 105 is also currently in the particular classification of data in the data storage device 389 in the cloud storage server 109.

[0066] In some embodiments, CPU 365 of management server 105-1 through 105-X is configured to implement the method of Figure 4, including any optional and / or auxiliary operations associated with the method of Figure 4. In some embodiments, computer-executable program instructions for performing operations of the method of Figure 4 are stored in computer memory 323 of management server 105-1 through 105-X.

[0067] As described herein, users are provided with access to play online video games within the cloud gaming system 100. Access to the cloud gaming system 100 can be provided across a wide geographic region. Cloud computing is a computing paradigm in which dynamically scalable, often virtualized resources are provided as a service over the Internet. Users do not need to be experts in the technical infrastructure of the computing "cloud" that supports them. Cloud computing can be categorized into different services, such as infrastructure as a service (IaaS), platform as a service (PaaS), and software as a service (SaaS). Cloud computing services often offer common online applications, such as video games, accessed through a web browser, but the software and data are stored on server computing systems within the cloud. The term cloud is used as a metaphor for the Internet based on how the Internet is depicted in computer network diagrams and is an abstract concept that hides a complex infrastructure.

[0068] In other embodiments, an online video game may be executed by a distributed game engine. In these embodiments, the distributed game engine may execute on multiple processing entities, such as cloud game servers 103-1 through 103-N, such that each processing entity executes a given functional segment of the game engine on the online host where the video game is executed. Each processing entity is viewed by the game engine as simply a computational node. A game engine typically performs a functionally diverse set of operations to execute a video game application along with additional services experienced by the user. For example, the game engine implements game logic and performs game calculations, physics, geometry transformations, rendering, lighting, shading, audio, and additional in-game or game-related services. Additional services executed by the game engine may include, for example, messaging, social utilities, audio communication, gameplay playback functions, help functions, etc. While a game engine may run on an operating system virtualized by a hypervisor on a particular server, in other embodiments, the game engine itself may be distributed across multiple processing entities, each residing on a different server unit in a data center. In some embodiments, the multiple processing entities include one or more of server units, virtual machines, and containers, among other types of processing entities, as needed for each game engine segment. For example, if a game engine segment is responsible for camera transformations, that particular game engine segment may be provisioned with a virtual machine associated with a graphics processing unit (GPU) because it will be performing a large number of relatively simple mathematical operations (e.g., matrix transformations). Also, by way of example, other game engine segments requiring fewer but more complex operations may be provisioned with processing entities associated with one or more more powerful central processing units (CPUs). In some embodiments, a single cloud gaming server 103-1 can host multiple light games (weighted in terms of computational resource consumption) for different users. In these embodiments, the cloud gaming server 103-1, including the storage CPU 331 and other components, is shared. This sharing of the cloud gaming server 103-1 can result in different levels of priority for storage requests and / or load balancing, as network / PCIe connections are shared and each user needs to get a fair amount of time. In some embodiments, a single cloud gaming server 103-1 can host both light games and heavier games (weighted in terms of computational resource consumption).

[0069] By distributing the game engine, the game engine has elastic computational characteristics that are not bound by the capabilities of a physical server unit. Instead, the game engine is provisioned with more or fewer computational nodes as needed to meet the demands of the online video game. From the perspective of the online video game and the video game player, a game engine that is distributed across multiple computational nodes is indistinguishable from a non-distributed game engine running on a single processing entity, because a game engine manager or supervisor distributes the workload and seamlessly integrates the results to provide the online video game output components to the player.

[0070] In some embodiments, users / players access remote services provided by cloud gaming system 100 through a client device, such as local computing system 120, which includes a processor, a display, and input / output (I / O) devices. In various embodiments, the client device may be a personal computer, a mobile phone, a netbook computer, a tablet computer, a personal digital assistant device, or another type of computing device. In some embodiments, a NIC running on cloud gaming servers 103-1-103-N recognizes the type of client device being used by the client and adjusts the data communication method between cloud gaming servers 103-1-103-N and the client device as needed. In some embodiments, the client device accesses cloud gaming servers 103-1-103-N over the Internet using standard communication methods, such as HTML (Hypertext Markup Language), TCP / IP (Transmission Control Protocol / Internet Protocol), or WebRTC (Web Real-Time Communication).

[0071] It should be understood that a given online video game or game application may be developed for a particular platform and a particular associated controller device. However, if such an online video game is made available via cloud gaming system 100 as described herein, a user may access the online video game with a different controller device. For example, a video game may be developed for a gaming console and its associated controller, but a user may access a cloud-based version of the video game from a personal computer using a keyboard and mouse. In such a scenario, an input parameter configuration may define a mapping from inputs that can be generated by a controller device available to the user (in this case, a keyboard and mouse) to inputs that are acceptable for execution of the cloud-based version of the video game.

[0072] In another example, a user may access the cloud gaming system 100 via a tablet computing device, a touchscreen smartphone, or other touchscreen-driven device. In this example, the client device and the controller device are integrated together within the same device, with input provided through detected touchscreen input / gestures. In such a device, an input parameter configuration may define specific touchscreen inputs corresponding to game inputs for a video game. For example, buttons, directional pads, and / or other types of input elements may be displayed or overlaid during the execution of an online video game to indicate locations on the touchscreen that a user can touch to generate game input. Gestures, such as swipes in specific directions or specific touch movements, may also be detected as game input. In one embodiment, to familiarize a user with control operations on a touchscreen, a tutorial showing how to input gameplay inputs via the touchscreen may be provided to the user, for example, before beginning gameplay of an online video game.

[0073] In some embodiments, the client device serves as a connection point for the controller device. More specifically, the controller device communicates with the client device via a wireless or wired connection and transmits inputs from the controller device to the client device. The client device may then process these inputs and then transmit the input data to cloud gaming servers 103-1 through 103-N over network 123, which may be accessed via a local network device such as a router. However, in another embodiment, the controller itself may be a networked device capable of communicating inputs directly to cloud gaming servers 103-1-103-N over network 123, without having to communicate such inputs through the client device first. For example, the controller may connect to a local network device (such as the router mentioned above) to send and receive data to and from cloud gaming servers 103-1-103-N. Thus, while the client device may still be required to receive video output from the cloud-based video game and render it on a local display, input latency can be reduced by allowing the controller to send inputs directly over the network to cloud gaming servers 103-1-103-N, bypassing the client device.

[0074] In some embodiments, networked controller and client devices may be configured to transmit certain types of input directly from the controller to cloud gaming servers 103-1-103-N and other types of input via the client device to cloud gaming servers 103-1-103-N. For example, input whose detection does not rely on any additional hardware or processing apart from the controller itself may be transmitted directly from the controller to cloud gaming servers 103-1-103-N over network 123, bypassing the client device. Such input may include button input, joystick input, embedded motion detection input (e.g., accelerometer, magnetometer, and / or gyroscope input), among other types of input. However, inputs that utilize additional hardware or require processing by the client device may be sent from the controller to the client device and then from the client device to the cloud gaming servers 103-1-103-N. These inputs may include captured video or captured audio from the game environment, which may require processing by the client device before transmission to the cloud gaming servers 103-1-103-N. Additionally, inputs from the controller's motion detection hardware may be processed by the client device in conjunction with the captured video to detect the position and motion of the controller, which is then communicated by the client device to the cloud gaming servers 103-1-103-N. It should be understood that controller devices according to various embodiments may also receive data (e.g., feedback data) from the client device or directly from the cloud gaming servers 103-1-103-N.

[0075] Embodiments of the present disclosure may be practiced with a variety of computer system configurations including handheld devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, etc. Embodiments of the present disclosure may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a wire-based or wireless network.

[0076] Although some method operations have been described in a particular order, it should be understood that other housekeeping operations may be performed between method operations and / or method operations may be arranged to occur at slightly different times or may be distributed across a system in which processing operations may occur at various intervals relative to the process, so long as the processing of the method operations provides for successful implementation of the method.

[0077] Some embodiments disclosed herein may also be fabricated as computer-readable code on a computer-readable medium. A computer-readable medium is any data storage device that can store data, which can then be read by a computer system. Examples of computer-readable media include hard drives, network-attached storage (NAS), read-only memory, random-access memory, CD-ROMs, CD-Rs, CD-RWs, magnetic tape, and other optical and non-optical data storage devices. In some embodiments, the computer-readable medium may include tangible computer-readable media distributed over a network-coupled computer system such that the computer-readable code is stored and executed in a distributed manner.

[0078] The foregoing description of the embodiments is provided for purposes of illustration and description and is not intended to be exhaustive or limiting. Individual elements or features of a particular embodiment are generally not limited to that particular embodiment, but are interchangeable where applicable and can be used in selected embodiments even if not specifically shown or described. As such, one or more features from one or more embodiments disclosed herein can be combined with one or more features from one or more other embodiments disclosed herein to form another embodiment not explicitly disclosed herein but implicitly disclosed herein. This other embodiment can also be modified in many ways. Variations of such embodiments should not be considered a departure from the disclosure herein, and all variations and modifications of such embodiments are intended to be within the scope of the disclosure provided herein.

[0079] Although the foregoing embodiments have been described in some detail for clarity of understanding, it will be apparent that certain changes and modifications may be practiced within the scope of the appended claims. Accordingly, the embodiments disclosed herein are to be considered illustrative and not restrictive, and the present embodiments should not be limited to the details set forth herein but may be modified within the scope of the appended claims and their equivalents.

Claims

1. A management server for a cloud gaming system, a root complex connected to a data link; a data bus connected to the root complex; a computer memory connected to the root complex; a network interface connected to the data bus; a processor connected to the data bus; the processor is configured to preload data associated with a control path application programming interface (API) call received over the data link before a data path API call for the data is received over the data link; A management server for a cloud gaming system, wherein the data is preloaded by receiving the data via the network interface and loading the data into the computer memory via the root complex and the data bus.

2. The management server for a cloud gaming system according to claim 1 , wherein the control path API call is an open API call.

3. The management server for a cloud gaming system according to claim 1 , wherein the control path API call is a mount API call.

4. The management server for a cloud gaming system of claim 1 , wherein the control path API calls are associated with the execution of a video game on the cloud gaming system.

5. the processor is configured, upon receipt of the control path API call, to execute the control path API call; The management server for a cloud gaming system of claim 4 , wherein execution of the control path API call blocks continued execution of the video game until execution of the control path API call is completed.

6. The management server for a cloud gaming system of claim 1 , wherein the data is preloaded from a data storage device that operates at a slower data rate than the computer memory.

7. The management server for a cloud gaming system according to claim 6 , wherein the data storage device is located outside the management server.

8. The management server for a cloud gaming system according to claim 6 , wherein the data storage device is a hard disk drive.

9. The management server for a cloud gaming system of claim 1 , wherein the computer memory is a non-volatile memory express solid state drive.

10. The management server for a cloud gaming system according to claim 1 , wherein the computer memory is a random access memory or a storage class memory.

11. The management server for a cloud gaming system of claim 1 , wherein the root complex includes an input / output memory management unit configured to map device-visible virtual memory addresses to physical memory addresses.

12. The management server for a cloud gaming system of claim 11 , wherein the input / output memory management unit is configured to provide the cloud gaming system with direct access to the data preloaded within the computer memory.

13. The management server for a cloud gaming system according to claim 1 , wherein the data link is a PCIe link.

14. The management server for a cloud gaming system according to claim 13 , wherein the root complex includes a PCIe port connected to the PCIe link.

15. 14. The management server for a cloud gaming system of claim 13, wherein the PCIe link enables the cloud gaming system to bypass the processor of the management server when accessing the computer memory within the management server.

16. 2. The management server for a cloud gaming system of claim 1, wherein the processor is configured to instruct data to be flushed from the computer memory through the network interface to a data storage device in a cloud storage system in response to receiving either an unmount control path API call or a commit control path API call over the data link.

17. the processor is configured to use a first network storage protocol for data communication on the data link; 2. The management server for a cloud gaming system of claim 1, wherein the processor is configured to use a second network storage protocol for data communication through the network interface, the second network storage protocol being different from the first network storage protocol.

18. The management server for a cloud gaming system according to claim 1 , wherein the data that the processor instructs to preload is a specific classification of data.

19. The management server for a cloud gaming system of claim 18 , wherein the particular classification of data is one or more of a data object, a data file, and a data block.

20. 20. The management server for a cloud gaming system of claim 18, wherein the particular classification of data is save disk images for video games running on the cloud gaming system.

Images