Edge computing proxy for cloud gaming and 5g

Abstract

A method is provided, the method comprising the operations of: executing a cloud video game in a data center; streaming a video generated by the executing cloud video game to a client device over a network; deploying a cloud game proxy to an edge compute proximate to the client device; wherein the cloud game proxy buffers the video and retransmits lost packets of the video to the client device.

Description

Technical Field

[0001] This disclosure relates to systems and methods for edge computing agents used in cloud gaming and 5G network communications. Background Technology

[0002] The current trend in the gaming industry is towards cloud gaming. Cloud gaming offers advantages to end users by enabling the remote execution of video games within data centers that guarantee the resources available for the games. The video generated by the remotely executed game is streamed to the user's device, and user input is sent back to the data center. This allows end users to run the game without owning specific hardware. Instead, end users only need sufficient hardware to stream gameplay and still enjoy a high-quality gaming experience. Furthermore, in theory, cloud gaming enables gaming from anywhere with available network connectivity.

[0003] To achieve portability and location freedom, game streaming via wireless networks is expected. However, wireless network communication can be problematic. For example, Wi-Fi connections can be unreliable due to interference or other connections using the same Wi-Fi frequency or channel. Alternatively, a user might be too far from the wirelessly connected device to receive a strong signal. These issues are likely to become even more pronounced with the rollout of 5G cellular networks, as more users may access the same base stations.

[0004] It is against this backdrop that the proposed implementation plan is put forward. Summary of the Invention

[0005] The implementation of this disclosure provides methods and systems for edge computing agents used in cloud gaming and 5G network communications.

[0006] In some implementations, a method is provided that includes the following operations: executing a cloud video game in a data center; streaming video generated by executing the cloud video game to a client device via a network; deploying a cloud gaming agent to an edge computing location close to the client device; wherein the cloud gaming agent buffers the video and retransmits lost packets of the video to the client device.

[0007] In some implementations, streaming the video, buffering the video, and retransmitting lost packets are performed in real time.

[0008] In some implementations, the retransmission of a given lost packet is in response to the failure to receive an acknowledgment of the given lost packet from the client device within a predefined time period following the previous transmission of the lost packet.

[0009] In some implementations, the retransmission of a given lost packet is in response to a request for the given lost packet from the client device.

[0010] In some implementations, the edge computing is defined at a wireless base station that facilitates wireless communication with the client device.

[0011] In some implementations, the wireless base station is a member of a cellular network.

[0012] In some implementations, the method further includes: deploying a second cloud gaming agent to a second edge computing facility, the second edge computing facility being defined at a second wireless base station near the wireless base station, the second cloud gaming agent receiving and buffering the video; and, in response to detecting that the client device switches from the wireless base station to the second wireless base station, activating the second cloud gaming agent to retransmit lost packets of the video to the client device.

[0013] In some implementations, the buffering of the video by the second cloud gaming agent and the buffering of the video by the cloud gaming agent occur simultaneously before the client device switches from the wireless base station to the second wireless base station.

[0014] In some implementations, the cloud gaming agent tracks video frames of the video and does not retransmit lost packets of the given video frame after a predefined time period has elapsed since the given video frame was transmitted to the client device.

[0015] In some implementations, the method further includes: streaming audio generated by executing the cloud video game to the client device via the network; wherein the cloud game agent buffers the audio and retransmits lost packets of the audio to the client device.

[0016] In some implementations, a non-transitory computer-readable medium is provided having program instructions that, when executed by a computing device, cause the computing device to perform a method comprising: executing a cloud video game in a data center; streaming video generated by executing the cloud video game to a client device via a network; deploying a cloud game agent to edge computing close to the client device; wherein the cloud game agent buffers the video and retransmits lost packets of the video to the client device.

[0017] Other aspects and advantages of this disclosure will become apparent from the following detailed description taken in conjunction with the accompanying drawings, which illustrate the principles of this disclosure by way of example. Attached Figure Description

[0018] This disclosure and its other advantages are best understood through the following description taken in conjunction with the accompanying drawings.

[0019] Figure 1 A system for streaming cloud video games to a client is conceptually illustrated according to an implementation of this disclosure.

[0020] Figure 2 A process for retransmitting data in cloud gaming streaming is conceptually illustrated according to an implementation of this disclosure.

[0021] Figure 3 A pairing process for deploying a cloud gaming agent, according to an implementation of this disclosure, is conceptually illustrated.

[0022] Figure 4 A conceptual illustration shows the migration of a cloud gaming agent between different base stations according to an implementation of this disclosure.

[0023] Figure 5 A cloud gaming agent configured to adjust the video or audio quality of a game stream is conceptually illustrated according to an implementation of this disclosure.

[0024] Figure 6A An exemplary system for loading game files of games available through a cloud gaming site, according to an implementation of this disclosure, is shown.

[0025] Figure 6B This is a flowchart conceptually illustrating, according to an implementation of the present disclosure, various operations performed to stream a cloud video game to a client device.

[0026] Figure 7 An implementation scheme of an information service provider architecture based on the implementation of this disclosure is shown. Detailed Implementation

[0027] Numerous specific details are set forth in the following description in order to provide a thorough understanding of this disclosure. However, it will be apparent to those skilled in the art that this disclosure may be practiced without some or all of these specific details. In other instances, well-known process steps have not been described in detail so as not to obscure this disclosure.

[0028] In a broader sense, implementations of this disclosure relate to interactive virtual / augmented characters that can be viewed through a head-mounted display. In some implementations, the virtual character is configured to move between a scene rendered on a display device in the local environment and the local environment itself as seen through the head-mounted display. In some implementations, the virtual character is capable of interacting with smart devices or IoT devices, such as by controlling these devices to turn on or off or perform other functions. In other implementations, various techniques are employed to ensure that the presence of the virtual character can be understood by other users in the local environment who cannot see the virtual character through the head-mounted display. For example, audio of the virtual character, such as the virtual character's voice, can be rendered through a local speaker system.

[0029] Broadly speaking, cloud gaming operates by streaming data from a data center to client devices via the internet. However, when using error-checking protocols (such as UDP networking), packet loss can cause audio / video artifacts on the client side. On the other hand, if error-checking protocols (such as TCP) are used and retransmission is required, this comes at the cost of increased latency, as packet retransmission takes additional time. Another strategy to address packet loss is to use forward error correction (FEC). However, FEC requires sending additional data, which consumes bandwidth.

[0030] However, most packet loss occurs close to the user, such as via their home WiFi connection or their LTE or 5G wireless connection. With the advent of 5G wireless technology, the availability of edge computing resources is also increasing, bringing computing to (or close to) cell towers. The implementation disclosed here employs a cloud gaming proxy on edge computing resources to achieve more stable and reliable video streaming.

[0031] 5G cell towers typically connect to the internet via fiber optic lines, and therefore their connectivity is generally quite fast and reliable. However, it is the last mile from the edge to the client, which can be unreliable. In the event of packet loss in cloud gaming applications, data currently needs to be retransmitted from data centers that are likely to be remote in terms of distance and latency. However, the implementation disclosed herein provides a cloud gaming agent deployed at an edge computing server to buffer any network packets between the client and the data center.

[0032] In cloud gaming applications, most data is transmitted from the data center (e.g., audio / video data) to the client. A cloud gaming agent can handle potential retransmissions back to the client (when needed). Especially in 5G wireless applications, ping times to cell towers and edge computing can be ~1ms, while ping times to the data center are typically around 30ms or more. However, by using a cloud gaming agent deployed at the edge to perform retransmissions, image quality and latency on the client side can be significantly improved because retransmissions become significantly faster (e.g., around 1ms).

[0033] Figure 1 A system for streaming cloud video games to a client, according to an implementation of this disclosure, is conceptually illustrated. In the illustrated implementation, cloud gaming system 100 is configured to execute video games for remote gaming by a user. Cloud gaming system 100 can be implemented in a data center via various hardware, such as one or more server computers. As shown, cloud gaming system 100 includes a game console 102 configured to execute a session of video game 104. Game console 102 can be a general-purpose or special-purpose computer, a game console, a blade version of a game console, a virtual machine, or any other computing resource capable of providing an operating system environment or platform on which video game 104 can be executed. In some implementations, game console 102 is a virtual machine executed in a data center via a hypervisor or cloud computing resources. It will be understood that during the execution of a video game session, input to the video game is processed, and gameplay video, including both image and audio data, is output for rendering at the client device.

[0034] In the illustrated implementation, user device 122 is a client device that receives video generated from the execution of video game session 104 via network 108 and renders the video for user viewing. By way of example only and not limitation, user device 122 may be a personal computer, laptop, tablet, cellular phone, portable gaming device, set-top box, game console, smart display, streaming stick or box, or any other device capable of receiving and rendering video of a video game for user viewing. It will be understood that in some implementations, user device 122 may include a display on which video is rendered for viewing, or may be operatively connected to such a display. User device 122 may execute client game application 124 configured to communicate with cloud gaming system 100 to stream video games from it, including receiving and rendering video output from the executing video game and facilitating interactive gameplay of the video game by the user using user device 122, and receiving input from the gameplay via network 108 and transmitting it back to the executing video game session 104. Such input can be generated from user operation of an input device that provides interactive input, which may be integrated into or separate from the user equipment 122, such as a touch screen, joystick, button, motion sensing hardware (e.g., accelerometer, gyroscope, magnetometer, etc.), camera, microphone, etc.

[0035] As noted, the video generated from the execution of video game session 104 is streamed over network 108 to user device 122 for rendering. In some implementations, streaming server 106 handles the streaming of the video and / or may further process the video before it is transmitted to user device 122. As an example and not a limitation, streaming server 106 may adjust video quality (such as its bitrate) based on network conditions, transcode the video to a suitable format, enlarge or reduce the video, or perform other adjustments as necessary to optimize video quality for user device 122.

[0036] In a broader sense, the implementation of this disclosure envisions user equipment 122 accessing a wireless data connection in order to stream gameplay from cloud gaming system 100. That is, the last hop or near-last hop from the cloud gaming system to user equipment 122 is via a wireless network. As an example, user equipment 122 may be a cellular device connected to a cellular data network. Alternatively, in other implementations, user equipment 122 is connected to a local wireless network, such as a Wi-Fi network. Therefore, user equipment 122 is located at or near the edge of the network topology between the cloud gaming system 100 (typically in a remote data center) and user equipment 122 via its wireless network for receiving and transmitting data.

[0037] As shown in the illustrated implementation, wireless base station 110 provides wireless network connectivity for wireless data transmission. In some implementations, wireless base station 110 is a cellular base station, thereby providing cellular network connectivity, such as 4G or 5G cellular networks. In other implementations, wireless base station 110 is a wireless router or other wireless networking device. For ease of description, wireless base station 110 is shown as separate from network 108. However, wireless base station 110 can be considered as part of network 108, specifically located at the edge of network 108, facilitating communication between cloud gaming system 100 and user equipment 122. As shown, wireless base station 110 includes transceiver 118 and antenna 120.

[0038] Therefore, in the illustrated implementation, video generated from video game session 104 is transmitted wirelessly via base station 110 to user equipment 122 for presentation via a display and speakers. In the event of data loss or packet loss during the transmission of video data or other video game-related data to user equipment 122, existing systems require retransmission of such data or packets from cloud gaming system 100. This requires communication all the way back to cloud gaming system 100 (or data center) via network 108, followed by retransmission to user equipment 122 again via the network. The amount of time required for such retransmission can be excessively long and may be infeasible for providing high-quality video for a real-time gaming environment. For example, retransmission may be impossible for high frame rate video because the time required for round-trip communication with cloud gaming system 100 exceeds the time required to render the next frame at the client.

[0039] To address these issues, a cloud gaming proxy 114 is provided at the edge to handle retransmissions as needed. The cloud gaming proxy is located at or near the wireless base station 110, close to the user equipment 122. In this way, retransmission of data or packets does not require communication all the way back to the cloud gaming system's data center, but only back to the wireless base station 110. The cloud gaming proxy 114 includes buffering logic 116 configured to buffer video or data packets and / or other data generated from the video game session 104 and transmitted to the client. Then, in the event of packet loss requiring partial retransmission to the user equipment 122, such packets can be retrieved from the buffer by the cloud gaming proxy and retransmitted to the user equipment 122 very quickly, as the cloud gaming proxy 114 is located at or near the base station 110 and thus close to the user equipment 122 throughout the network topology. Most of the data transmitted to user device 122 is typically used for gameplay videos, and by buffering the gameplay videos at the edge of the network close to user device 122, such data / packets can be retransmitted very quickly even if the wireless connection causes data loss or packet loss, so as to minimize the latency impact of video games and enable the maintenance of high-fidelity video presentation.

[0040] In some implementations, the cloud gaming agent 114 acts as a path point, enabling gameplay videos to be streamed to the cloud gaming agent 114, which then buffers the gameplay videos and transmits them to the user device 122.

[0041] In another implementation, cloud gaming proxy 114 intercepts and buffers data packets addressed to the user device for gameplay videos. In this implementation, cloud gaming proxy 114 also intercepts acknowledgments from the user device, as described in further detail below. In this way, cloud gaming proxy 114 is transparent to the user device while accelerating packet retransmission.

[0042] The cloud gaming agent 114 can be implemented on edge computing resource 112. Broadly speaking, edge computing resource 112 is a available processing resource at or near base station 110 for executing programs or applications. In some implementations, the edge computing resource is implemented as an edge computing server or edge node accessible to deploy programs / applications at base station 110. In some implementations, the edge computing resource is a mobile edge computing or multiple access edge computing (MEC) application server providing computing and storage resources, and may further provide access to real-time radio access network (RAN) information.

[0043] It will be understood that communication between user device 122 and cloud gaming agent 114 may utilize standard protocols (e.g., UDP, TCP / IP, etc.) or non-standard or custom protocols.

[0044] Figure 2 A process for retransmitting data in a cloud gaming streaming operation, according to an implementation of this disclosure, is conceptually illustrated. The executing cloud video game continuously generates video frames, as indicated by reference numeral 200. By way of example and not limitation, a representative video frame 202 is shown in the illustrated implementation. Video frame 202 data (which may be encoded according to a video codec / format or video compression protocol) is packetized (reference numeral 204) for transmission over a network (e.g., the Internet). The packets are transmitted to a cloud gaming agent at the edge of the network, which buffers the packets as they are transmitted to the client, storing them in buffer 206. A frame assembly process 212 is performed at the client, thereby defining that the packets of video frame 202 are unpacked and assembled (and optionally decoded) to enable the replication of video frame 202 at the client. Ideally, the packets received at the client include all the data of the video frame and arrive on time, so that the video frame 202 can be rendered 216 onto a display 218 in a manner that faithfully reproduces the original video frame 202.

[0045] However, packet loss may occur from the base station at the edge to the client, and therefore, as described herein, packet retransmission may occur at the edge to minimize the amount of time required for retransmission. In the case of positive acknowledgment error-checking communication protocols (such as TCP / IP), packets are checked when received at the client, and a positive acknowledgment is sent for packets received without errors. Typically, such positive acknowledgments must be sent all the way back to the originating data center, and unacknowledged packets are automatically retransmitted after a given period of time. However, according to the implementation of this disclosure, positive acknowledgments are sent to cloud gaming agent 114, and the cloud gaming agent can therefore be configured to automatically retransmit a given packet that has not been positively acknowledged after a given period of time (e.g., approximately the expected time for transmitting the packet and receiving the corresponding positive acknowledgment) (reference numeral 210).

[0046] In some implementations, a negative acknowledgment framework is used, such that if an error is detected or the client does not receive the expected packet within the expected timeframe, a negative acknowledgment is sent for the relevant packet. This serves as a request to retransmit the relevant packet from the cloud gaming agent (reference numeral 208), and the required packet is retransmitted from buffer 206 (reference numeral 210). As noted, this does not require communication all the way back to the cloud gaming system, but rather processing occurs at the edge of the network, resulting in very fast round-trip times.

[0047] Generally, to provide a high-quality video streaming experience, the rendering of video frames at the client should be "timely," meaning it should be based on their expected timing within the sequence of video frames that define the gameplay video. For example, if video frames are generated at 60 frames per second (fps), then rendering such video frames at the client should also be done at 60 fps, but with some delay / offset relative to the original video frame generation. Furthermore, even if the frame rate is to change constantly (variable frame rate), the timing of rendering such video frames at the client should match these changes, ensuring that video frames are rendered according to their original timing, with only the necessary delay caused by the time required to stream the video over the network to the client. In other words, the latency from the cloud gaming system to the client should be consistent to ensure smooth video rendering at the client that matches the timing of video frame generation at the cloud gaming system.

[0048] Therefore, in some implementations, the timing of the given frame to which the packet belongs is considered before deciding whether to retransmit a lost packet from the cloud gaming agent (according to a positive acknowledgment framework) or to request a retransmission of a given packet from the client (according to a negative acknowledgment framework). If a packet is lost or malfunctions in some way, but there is not enough time to retransmit it before rendering the frame to which the packet belongs, the cloud gaming agent does not retransmit the packet, or the client does not send a request for retransmission. If there is enough time, the cloud gaming agent retransmits the packet, or the client sends a request for retransmission. It will be understood that determining whether there is enough time can be based on a predefined amount of time or a deadline. For example, if the time to render a video frame is less than a predefined amount of time, the packet associated with that video frame is not retransmitted or a request for retransmission of that packet is not sent; otherwise, the packet is retransmitted or a retransmission request is sent.

[0049] Therefore, in some implementations, the packet header identifies which video frame the data in the packet belongs to, and may also identify the timing of the video frame. This information is then used to determine whether to perform a packet retransmission or a request for packet retransmission, as described above. For example, in some implementations of the positive acknowledgment framework, the cloud gaming agent tracks the timing and sequence of video frames as it continuously receives them from the data center, such that retransmission of a given video frame packet stops after a predefined amount of time has elapsed since the first transmission of the video frame to the client. It will be understood that the predefined amount of time may be based on the expected time for rendering the video frame at the client.

[0050] In some implementations, error-free communication protocols (e.g., UDP) are used to transmit packets over the network. In this case, packets may include frame identifier information, but it may be impossible to determine whether a packet has been lost until the video frame is assembled from available packets that have already been received, or until a count / count of packets received for a video frame is performed at a given time point (e.g., at predefined intervals). Therefore, in some implementations, an error checking process 214 is performed, which may be based on frame assembly process 212 or a portion thereof, to determine whether a given video frame is missing data, and if so, a request for such data is sent back to the cloud gaming agent, and the relevant data is retrieved from buffer 206 and retransmitted to the client. It will be understood that the timing of frame rendering at the client can be applied to determine whether data for a given video frame is requested, similar to the above. In other words, depending on whether there is sufficient time before the video frame is rendered at the client, a request for data retransmission may or may not be sent.

[0051] In some implementations, graphic fill techniques can be used at the client-side to fill in missing pixel data in video frames. For example, such graphic fill techniques could be AI-based graphic generation or magnification techniques that use AI to generate or adjust graphics based on the existing graphic context.

[0052] It will be understood that the buffering of video for cloud gaming provided by the cloud gaming agent described herein differs from existing systems that cache or buffer pre-existing content at the edge of the network. For pre-existing content (such as movies, music, or web pages), they can be cached well before retrieval or even upon request. "Live streaming" content can even be delivered with significant latency (e.g., greater than 500ms to several seconds) without impacting the user experience. However, cloud gaming video content generated in real-time must be streamed in real-time to provide a high-quality gaming experience. In various implementations, real-time video delivery can be defined as the duration or latency from the rendering of video frames by executing the video game at a data center to the presentation of video frames by the client device being less than approximately 200ms, less than approximately 150ms, less than approximately 100ms, less than approximately 50ms, less than approximately 30ms, etc. Therefore, the cloud gaming agent of this implementation buffers streaming gameplay video in real-time, thereby enhancing real-time video delivery and gameplay.

[0053] Figure 3 A pairing process for deploying a cloud gaming agent according to an implementation of this disclosure is conceptually illustrated. Broadly speaking, the cloud gaming system identifies the base station to which the user equipment is connected in order to determine the location for deploying the cloud gaming agent. In the illustrated implementation, base station 110 has a base station identifier 300, which is a unique identifier associated with base station 110. In some implementations, the base station identifier 300 may be a base station identification code, a GSM cell ID, an IP address, a fully qualified domain name (fqdn), or some other form of identification information uniquely assigned to base station 110.

[0054] User equipment 122 can access base station identifier 300 when connected to base station 110. When initiating a cloud gaming session, user equipment 122 transmits base station identifier 300 to cloud gaming system 100. Cloud gaming system 100 then uses base station identifier 300 to identify base station 110 and further deploys cloud gaming agent 114 at base station 110. In this way, cloud gaming system 100 can deploy cloud gaming agent 114 at base station 110 that user equipment 122 is using.

[0055] In some implementations, the edge node at base station 110, on which the cloud gaming agent 114 is deployed, has an IP address. The IP address of the edge node can be obtained using base station identifier 300, such as by accessing a lookup table. The cloud gaming system 100 can then communicate with the IP address of the edge node, and the edge node can communicate with the IP address of the client.

[0056] It will be understood that if user equipment 122 switches to a different base station, that base station will also have a unique base station identifier, which can be communicated back to the cloud gaming system, allowing the cloud gaming agent to be redeployed to the new base station. In this way, the cloud gaming agent can be maintained at the base station closest to and used by user equipment 122.

[0057] Generally speaking, the cloud gaming agent 114 is expected to be a lightweight program, so that a relatively small amount of code (e.g., less than 1MB) can be deployed at base station 110.

[0058] Broadly speaking, when a user engages in interactive gameplay of a cloud video game, the user can operate or interface with one or more controller devices to generate input for the video game. By way of example only and not limitation, such devices may include one or more of a keyboard, mouse, touchpad, trackball, game controller, motion controller, joystick, button, trigger, camera, microphone, etc. In some implementations, input generated by operating such controller devices is relayed back to the cloud video game via cloud gaming agent 114. However, in other implementations, input is relayed to the cloud video game separately from cloud gaming agent 114 (e.g., via an alternative data channel).

[0059] In some implementations, cloud gaming agents are configured to buffer data for up to approximately 100 milliseconds; in others, approximately 10 to 50 milliseconds (e.g., video data); and in still others, approximately 20 to 30 milliseconds. For example, this might require buffering up to approximately 0.1 MB of data for a 1080p video stream at approximately 10 megabits per second, or up to approximately 0.6 MB of data for a 4K video stream at approximately 50 megabits per second.

[0060] Figure 4 This paper conceptually illustrates the migration of a cloud gaming agent between different base stations according to an implementation of this disclosure. In the illustrated implementation, at an initial time point, a user at location L1 streams a cloud video game from data center 400 via a local cellular tower (or base station) T1. That is, the user's device, which is operating to access and stream the cloud video game, is located at location L1 and communicates via a connection to tower T1. Therefore, a cloud gaming agent is deployed at tower T1 to buffer the video stream at tower T1.

[0061] However, due to situations such as tower T1 becoming too busy (e.g., too many users accessing tower T1, reducing the bandwidth available to users) or users moving to different locations, it may be necessary to switch towers. For example, a user may switch from access tower T1 to access tower T2. In this case, the cloud gaming agent migrates from tower T1 to tower T2. For example, when a user device switches from using tower T1 to using tower T2, the base station identifier of tower T2 is communicated to data center 400, and the cloud gaming agent is migrated or redeployed to tower T2. In some implementations, the cloud gaming agent is redeployed from the data center and re-instantiated at tower T2. In some implementations, the cloud gaming agent is copied or moved from tower T1 to tower T2. That is, in some implementations, the cloud gaming agent at tower T1 will be instantiated as another cloud gaming agent at tower T2.

[0062] In some implementations, existing buffered data is transmitted from tower T1 to tower T2, while in others, buffered data is not transmitted. In some implementations, gameplay is paused during the migration or redeployment of the cloud gaming agent. That is, when a cloud gaming agent migration occurs, a signal is sent to the cloud video game to pause its execution. In some implementations, a pause or interruption screen or other notification may be displayed when this occurs. In some implementations, when the cloud gaming agent migration is complete and the system is ready to resume game streaming, the user may be presented with options to resume gameplay.

[0063] However, where possible, it is generally desirable to avoid gameplay interruptions, and therefore, in some implementations, to prevent video game interruptions, the cloud gaming agent at tower T2 is instantiated before the user device switches from tower T1 to tower T2. Thus, for a given period of time, the data center can stream data to both towers T1 and T2, even while the user device remains connected to tower T1. Then, when the user device switches to tower T2 (and accordingly switches to using the cloud gaming agent at tower T2), the game streaming is uninterrupted because the data has already been streamed to tower T2 synchronously with the data being streamed to tower T1.

[0064] As will be understood, based on the above, a cloud gaming agent can be preemptively instantiated at a nearby tower based on several factors. As an example, and not a limitation, such factors may include bandwidth or streaming quality. For instance, if packet loss exceeds a given threshold, or if the video stream quality drops below a certain threshold, a cloud gaming agent can be preemptively instantiated at a nearby tower.

[0065] Another factor is the location of the user's device. For example, in the implementation shown, when a user is at location L1 and connected to tower T1, the user can be assigned to zone Z1 of a tower close to the user's location. The cloud gaming agent can be redundantly deployed at towers within the user's zone. In the implementation shown, zone Z1 includes towers T1, T2, and T3, and therefore when the user is at location L1 and specifically streaming gameplay through tower T1, the cloud gaming agent is also redundantly deployed at towers T2 and T3. By way of example and not limitation, in some implementations, a tower zone can include any tower within a predefined radius of the user's device location.

[0066] Continue to refer to Figure 4 When a user moves to location L2, they can be assigned to a different zone Z2 of a tower, which includes towers T2, T3, and T4. It will be understood that the user now accesses tower T2, and the transition is seamless because the cloud gaming agent was already deployed at tower T2 before the user moved to location L2. By proactively deploying the cloud gaming agent in a redundant manner in the background, smooth and uninterrupted game streaming can be maintained even when the user's device switches from one cellular tower to another.

[0067] In some implementations, user movement or trajectories can be tracked and used accordingly to determine which towers to deploy cloud gaming agents on. For example, a user's future location can be predicted based on their current movement or trajectory, and towers near the predicted future location can be identified and cloud gaming agents preemptively deployed on them. In some implementations, GPS tracking or other tracking technologies can be used to determine the location and movement of the user's device.

[0068] In some implementations, user equipment can perform ping tests or communication tests on the local cellular tower to determine signal strength, latency, bandwidth, or other network-related factors that may affect streaming quality. The results of such tests can be relayed back to the data center and used by the cloud gaming system to select the tower for deploying the cloud gaming agent. This information can be useful in selecting which tower is optimal, as the geographically nearest tower may not necessarily provide the best network conditions for game streaming (e.g., due to obstacles, interference, etc.).

[0069] Therefore, according to the implementation of this disclosure, video games can be streamed in parallel to multiple towers. The streamed data will have the same frame identifier or audio packet identifier, and the client can receive data from multiple towers and then decide to switch to another tower based on improved performance (e.g., lower latency), or in fact, decide not to switch towers if performance is not improved or if doing so would actually degrade performance (e.g., increased latency).

[0070] In some implementations, tower usage can be based on service tier / level relative to network service providers or cloud gaming providers. For example, users with higher service tiers may have higher priority in using a given tower (such as the nearest tower). That is, their game streaming and cloud gaming proxies will be prioritized over those with lower service tiers. Users with lower service tiers will have lower priority in using towers, and their streaming quality may be degraded or they may be switched to another tower when bandwidth through a given tower is insufficient.

[0071] Figure 5 A cloud gaming agent configured to adjust the video or audio quality of a game stream is conceptually illustrated according to an implementation of this disclosure.

[0072] In some implementations, the cloud gaming agent 114 may include a video adjuster 502 configured to adjust the video stream as needed based on changing network conditions. For example, a user may initially connect to a given cellular tower and stream in 4K. However, the user may switch to a tower that can only handle 720p streaming. In some implementations, the cloud gaming agent's video adjuster 502 is configured to downsample or transcode the video to suit available network conditions; that is, to change the video to a format or settings that require less bandwidth. For example, the video may be downsampled to 720p by the cloud gaming agent 114 before being transmitted to the user's device. In some implementations, the downsampled video may be processed by the client by applying different settings. For example, the client game application 124 may apply upscaling to the video that has been downsampled by the cloud gaming agent.

[0073] Similarly, the audio portion of the video can be adjusted by the cloud gaming agent 114's audio adjuster 504 based on network conditions. For example, the audio can be adjusted from 5.1 surround sound or mixed to stereo or mono audio. That is, the audio can be changed to a lower bandwidth format or settings.

[0074] It will be understood that such adjustments can be performed based on the quality of the game streaming detected by the streaming monitor 500. For example, streaming feedback data 506, which indicates the quality of the game stream, can be transmitted to the streaming monitor 500. As an example and not a limitation, if packet loss or data loss at user device 122 exceeds a threshold, the streaming monitor 500 can trigger the video tuner 502 and / or the audio tuner 504 to downsample or otherwise adjust the video or audio stream to require less bandwidth. In some implementations, the detection of such conditions can occur at the cloud gaming agent level. For example, in some implementations, the streaming monitor 500 can monitor retransmission requests from the client game application 124, and if the cloud gaming agent receives requests for retransmission packets / data at a rate exceeding a predefined threshold, this can indicate low connection quality, and the video tuner 502 and / or the audio tuner 504 can therefore be invoked to adjust the video / audio streaming to reduce bandwidth requirements. The adjusted audio / video data 508 is transmitted to the client game application 124 and processed accordingly for rendering by the user device 122.

[0075] In some implementations, cloud gaming agents deployed at edge computing are ephemeral processes. For example, edge computing at cellular towers can be leased as needed, and therefore, cloud gaming agents are instantiated when needed and terminated when no longer needed. In other implementations, cloud gaming agents are assigned based on the lifecycle of signals from clients, such that the process terminates if no signal is received from the client within a predefined amount of time.

[0076] The implementations disclosed herein may be included as part of a game engine. Broadly speaking, a game engine is a software development framework that provides features for the efficient development of video games. A game engine may include a software library with reusable modules to handle various aspects of game functionality, including, but not limited to, graphics rendering (e.g., including vertex processing, polygon processing, shading, lighting, texturing, etc.), sound, physics (including collision handling), animation, scripting, artificial intelligence, networking, streaming, memory management, thread handling, localization support, scene graphics, cutscenes, etc.

[0077] Game engines can be optimized for different hardware platforms, such as game consoles, mobile devices, and personal computers. As an example, and not a limitation, a game engine can optimize memory usage based on the platform (e.g., how to prioritize various tasks in the graphics pipeline). In some implementations, the hardware can be a blade version of a specific processing entity, such as a game console. Therefore, a user can be assigned to a specific blade that provides the same hardware that the console game has already optimized for.

[0078] It will be understood that game server logic may also exist to provide streaming and / or other services (packetization, encoding, Quality of Service (QoS) monitoring, bandwidth testing, access to social networks / friends, etc.).

[0079] In some implementations, the cloud infrastructure can run a hypervisor that abstracts the hardware and provides a virtual machine framework on which an operating system (OS) can be loaded. Therefore, the stack can include applications / video games running on the OS, which is loaded onto a virtual machine (VM) instantiated by the hypervisor, which is loaded onto the underlying hardware. In this way, application execution is not necessarily coupled to specific hardware.

[0080] In some implementations, applications / video games can be executed through containers that are abstracted at the application layer, thereby bundling code and dependencies together, enabling OS or hardware platform agnostic software development, and facilitating cross-platform software portability.

[0081] In some implementations, a distributed game engine is employed, where different parts of the game engine can be handled by different computational entities. For example, game engine functions such as physics engine, rendering engine (2D / 3D graphics), sound, scripting, animation, AI, networking, streaming (encoding), memory management, and thread processing can be divided into different functional processing blocks and / or services distributed across many different computations. It will be understood that for distributed game engines, low-latency communication is required to avoid latency issues. To maintain the desired frame rate, the total time for computation and communication must meet certain constraints. Therefore, dividing certain tasks may or may not be efficient, depending on whether it is possible to complete the process in a shorter time.

[0082] The advantage of using a distributed game engine is the ability to leverage elastic computing, where computing resources can be scaled up or down as needed. For example, in large multiplayer games traditionally run on a single hardware server, after, say, about 100 players, hardware resources become limited, making it impossible to add more players. The game might then have additional players queuing, meaning players have to wait to join. However, in the case of a distributed game engine, by using elastic cloud computing resources, more computing nodes can be added to meet the demand, thus supporting, for example, thousands of players. The game is no longer limited by a specific hardware server.

[0083] Therefore, cloud gaming engines can distribute functionality across different processing entities. It will be understood that different functions can be executed in different frameworks. For example, some functions (e.g., social) might be easier to run in containers, while graphics might run better using VMs connected to the GPU.

[0084] To facilitate the distribution of cloud gaming engine functionality, a distribution / synchronization layer manages job distribution, such as sending out jobs, receiving back data, identifying which tasks to perform and when to perform them, and handling queuing (e.g., if jobs complete faster than required). In some implementations, a given task can be dynamically subdivided if needed. For example, animation might have lighting, and if the lighting is particularly complex, it could be subdivided into three lighting jobs, which are sent out for computation and reassembled upon return. Therefore, if game engine functionality requires more work, it can be further subdivided.

[0085] Cloud service providers offer computing at a specified performance level (e.g., in terms of input / output operations per second (“IOPS”). Therefore, game providers can specify VMs, dedicated processing power, storage capacity, etc., from the cloud service provider and instantiate a distributed cloud game engine using the cloud service provider's system.

[0086] In some implementations, the library modules and update handlers can be one or more components or modules of the game engine. In some implementations, the library modules and update handlers can be separate components or integrated. In some implementations, the library modules and update handlers can operate as supplements to the game engine. In some implementations, the game engine can be a distributed game engine, as noted above.

[0087] As noted, the implementation of this disclosure can be applied to cloud gaming systems. An example of a cloud gaming system is... Cloud gaming systems. In such systems, the client device can be a game console, such as... The game console, or another device such as a personal computer, laptop, tablet, cellular phone, or mobile device.

[0088] Broadly speaking, to implement cloud gaming, when a user request for a game name is received, one or more servers perform several operations within a data center associated with the cloud gaming site. When the cloud gaming site receives a user request, it identifies the data center hosting the game associated with the selected game name and sends the request to the identified data center to instantiate the game for the selected game name. In response to the request, the server at the data center identifies the game code, loads the identified game code, and initializes the files associated with the game code to prepare for presenting the game content to the user. Game data associated with the game may include general game data and user-specific game data. Therefore, initializing the files may include identifying, loading, and initializing both general game data and user-specific game data. Initializing general game data may include initializing the graphics engine, installing graphics data, initializing sound files, installing original graphics, etc. Initializing user-specific data may include locating, transferring, and installing user data, user history, game history, etc.

[0089] During the loading and initialization of general game data, a "launch" screen can be provided for rendering on the client device. The launch screen may be designed to provide a representative image of the game being loaded, allowing the user to preview the type of game being loaded. Once the general game data is loaded, some initial content can be rendered, and a selection / navigation screen can be presented for user selection and customization. User input provided on the selection / navigation screen may include game level selection, game icon selection, game mode selection, game rewards, and other user-related data that may require the upload of additional game content. In some implementations, game content is made available for viewing and interaction by streaming game content from a game cloud system to the user's computing device. In some implementations, game content is available for gameplay after loading user-specific data.

[0090] Figure 6AAn exemplary system for loading game files of games available through a cloud gaming site is illustrated. The system includes multiple client devices 600 communicatively connected to a cloud gaming site 604 via a network 602, which may include a LAN network, wired network, wireless network, cellular network (e.g., 4G, 5G, etc.), or any other type of data network, including the Internet. When a request to access the cloud gaming site 604 is received from a client device 600, the cloud gaming site 604 accesses user account information 606 stored in a user data storage area 608 to identify the user associated with the client device that initiated the request. In some embodiments, the cloud gaming site may also verify the identified user to determine all games that the user is authorized to view / play. After user account identification / verification, the cloud gaming site accesses a game name data storage area 610 to identify game names available at the game cloud site for the requesting user account. The game name data storage area 610 then interacts with a game database 612 to obtain game names for all games available at the cloud gaming site. When a new game is launched, the game database 612 is updated with the game code, and the game name information of the newly launched game is provided to the game name data storage area 610. When a request is made, the requesting client device may or may not register with the cloud gaming site. If the user of the requesting client device is not a registered user, the cloud gaming site may identify the user as a new user and select a suitable game name for the new user (e.g., a default set of game names). The identified game name is returned to the client device for display on screen 600-a, as shown. Figure 6A As shown.

[0091] User interaction is detected at one of the game names rendered on the client device, and a signal is sent to the cloud gaming site. This signal includes information about the game name where the user interaction was detected, as well as the user interaction registered at that game name. In response to the signal received from the client device, the cloud gaming site proactively determines the data center hosting the game and sends a signal to the identified data center to load the game associated with the game name where the user interaction was detected. In some implementations, more than one data center may be hosting the game. In such implementations, the cloud gaming site may determine the geographic location of the requesting client device, identify a data center geographically close to the client device, and signal that data center to preload the game. The user's geographic location can be determined using a Global Positioning System (GPS) mechanism within the client device, the client's IP address, the client's ping information, etc. Of course, the aforementioned methods of detecting the user's geographic location are exemplary, and other types of mechanisms or tools can be used to determine the user's geographic location. Identifying a data center close to the client device minimizes latency during user interaction with the game. In some implementations, the identified data center may not have the bandwidth / capacity required to host the game or may be overused. In these implementations, the cloud gaming site may identify a second data center geographically close to the client device. Game loading includes loading the game code and executing an instance of the game.

[0092] In response to receiving a signal from a cloud gaming site, the identified data center can select a server at that data center to instantiate the game on. The server selection is based on available hardware / software capabilities and the game's requirements. The server may include multiple game consoles, and the server can determine which of these consoles to use to load the game. The game console may resemble a standalone game console, or it may be a rack-mounted server or a blade server. A blade server may further include multiple server blades, each with the circuitry required to instantiate a single, dedicated application, such as a game. Of course, the game consoles described above are exemplary and should not be considered limiting. Other types of gaming consoles (including gaming stations, etc.) and other forms of blade servers may also be used to host the identified game.

[0093] Once the game console is identified, the game's generic game-related code is loaded onto it, and a signal is sent back to the client device via the cloud gaming site, thus identifying the game console on which the game is instantiated. The loaded game is then available to the user.

[0094] Figure 6BThis is a flowchart conceptually illustrating various operations performed to stream a cloud video game to a client device, according to an implementation of this disclosure. Game system 618 executes the video game and generates raw (uncompressed) video 620 and audio 622. Video 620 and audio 622 are captured and encoded for streaming purposes, as indicated by reference numeral 624 in the illustrated figures. Encoding provides compression of the video and audio streams to reduce bandwidth usage and optimize the gaming experience. Examples of encoding formats include H.265 / MPEG-H, H.264 / MPEG-4, H.263 / MPEG-4, H.262 / MPEG-2, WMV, VP6 / 7 / 8 / 9, etc.

[0095] The encoded audio 626 and encoded video 628 are further packetized into network packets, as indicated by reference numeral 632, for transmission over a network (such as the Internet). The network packet encoding process may also employ data encryption to provide enhanced data security. In the illustrated implementation, audio packet 634 and video packet 636 are generated for transmission over the network, as indicated by reference numeral 640.

[0096] The game system 618 also generates haptic feedback data 630, which is further packetized into network packets for network transmission. In the illustrated implementation, haptic feedback packets 638 are generated for transmission over the network, as further indicated by reference numeral 640 in the accompanying drawings.

[0097] The aforementioned operations of generating raw video and audio data, encoding the video and audio, and packetizing the encoded audio / video and haptic feedback data for transmission are performed on one or more servers that jointly define the cloud gaming service / system. As indicated by reference numeral 640, the audio, video, and haptic feedback packets are transmitted over a network (such as and / or including the Internet). As indicated by reference numeral 642, audio packet 634, video packet 636, and haptic feedback packet 638 are decoded / reassembled by the client device to define encoded audio 646, encoded video 648, and haptic feedback data 650 at the client device. If the data is encrypted, the network packets are also decrypted. Then, as indicated by reference numeral 644, the client device decodes the encoded audio 646 and encoded video 648 to generate client-side raw audio and video data for rendering on display device 652. The haptic feedback data 650 can be processed / transmitted to generate haptic feedback effects at the controller device 656 or other interface devices through which haptic effects can be rendered. An example of a haptic effect is vibration or rumble from the controller device 656.

[0098] It will be understood that video games respond to user input and therefore can perform a process flow similar to the above-described process for transmitting and processing user input, but in the opposite direction from the client device to the server. As shown, the user-operated controller device 656 can generate input data 658. This input data 658 is packetized at the client device for transmission over the network to the cloud gaming system. The input data packet 660 is unpacked and reassembled by the cloud gaming server to define input data 662 on the server side. Input data 662 is fed to the game system 618, which processes the input data 662 to update the game state of the video game.

[0099] During the transmission of audio packet 634, video packet 636, and haptic feedback packet 638 (reference numeral 640), data transmission over the network can be monitored to ensure the quality of service for the cloud gaming stream. For example, as indicated by reference numeral 664, network conditions, including both upstream and downstream network bandwidth, can be monitored, and the game streaming can be adjusted in response to changes in available bandwidth. That is, the encoding and decoding of network packets can be controlled based on the current network conditions, as indicated by reference numeral 666.

[0100] Figure 7 An implementation scheme of an information service provider architecture is illustrated. Information service provider (ISP) 770 provides a wide range of information services to geographically dispersed users 782 connected via network 786. An ISP may deliver only one type of service, such as stock price updates, or it may deliver multiple services, such as broadcast media, news, sports, games, etc. Furthermore, the services provided by each ISP are dynamic, meaning that services can be added or removed at any point in time. Therefore, the ISP providing a specific type of service to a particular individual may change over time. For example, when a user is in her hometown, she may be served by an ISP very close to her, and when she travels to a different city, she may be served by a different ISP. The hometown ISP will transfer the necessary information and data to the new ISP, causing the user information to "follow" the user to the new city, thus making the data closer to the user and more easily accessible. In another implementation, a master-server relationship may be established between a master ISP that manages information for the user and a server ISP that directly interfaces with the user under the master ISP's control. In yet another implementation, when a client moves worldwide, data is transferred from one ISP to another, so that the ISP in a better location to serve the user becomes the ISP delivering these services.

[0101] ISP770 includes Application Service Provider (ASP)772, which provides computer-based services to customers via a network (e.g., including but not limited to any wired or wireless network, LAN, WAN, WiFi, broadband, cable, fiber optic, satellite, cellular (e.g., 4G, 5G, etc.), the Internet, etc.). Software provided using the ASP model is sometimes also called software-on-demand or Software as a Service (SaaS). A simple form of providing access to a specific application (such as customer relationship management) is using a standard protocol (such as HTTP). The application software resides on the vendor's system and is accessed by users via a web browser using HTML, via dedicated client software provided by the vendor, or via other remote interfaces (such as thin clients).

[0102] Services delivered over a wide geographical area typically utilize cloud computing. Cloud computing is a computing approach that delivers dynamically scalable and often virtualized resources as a service via the internet. Users do not need to be experts in the technical infrastructure supporting their “cloud.” Cloud computing can be categorized into different services, such as Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS). Cloud computing services typically provide common online business applications accessible through a web browser, while software and data are stored on servers. Based on how the internet is depicted in computer network diagrams, the term “cloud” is used as a metaphor for the internet (e.g., using servers, storage devices, and logic) and an abstract concept of the complex infrastructure it hides.

[0103] Furthermore, the ISP770 includes a game processing server (GPS) 774, which is used by game clients to play single-player and multiplayer video games. Most video games played on the internet run by connecting to a game server. Typically, games use a dedicated server application that collects data from players and distributes it to other players. This is more efficient and effective than a peer-to-peer arrangement, but it requires a separate server to host the server application. In another implementation, the GPS establishes communication between players, and the players' respective gaming devices exchange information without relying on a centralized GPS.

[0104] Dedicated GPS servers operate independently of the client. These servers typically run on dedicated hardware located in data centers, providing greater bandwidth and dedicated processing power. For most PC-based multiplayer games, dedicated servers are the preferred method for hosting game servers. Large-scale multiplayer online games run on dedicated servers, which are often hosted by the software company that owns the game's title, allowing them to control and update content.

[0105] A broadcast processing server (BPS) 776 distributes audio or video signals to viewers. Broadcasting to a very small audience is sometimes called narrowcasting. The final stage of broadcast distribution is how the signal reaches the listener or viewer, and it may travel through the air to an antenna and receiver, like a radio or television station, or it may be transmitted via cable television or wired broadcasting (or “wireless cable”) through a workstation or directly from a network. The Internet can also bring radio or television to the receiver, especially through multicasting, which allows for the sharing of signals and bandwidth. Historically, broadcasting has been defined by geographical areas, such as national broadcasting or regional broadcasting. However, with the widespread availability of the fast Internet, broadcasting is no longer defined by geographical conditions, as content can reach almost any country in the world.

[0106] Storage Service Providers (SSPs) 778 provide computer storage space and related management services. SSPs also offer regular backups and archiving. By offering storage as a service, users can subscribe to more storage as needed. Another major advantage is that SSPs include backup services, so users will not lose all their data if their computer's hard drive fails. Furthermore, multiple SSPs can have full or partial copies of user data, allowing users to access data efficiently, regardless of their location or the device used to access it. For example, a user can access personal files on their home computer and their mobile phone when they are on the move.

[0107] Communication providers 780 offer connectivity to users. One type of communication provider is an Internet Service Provider (ISP) that provides access to the Internet. ISPs use data transmission technologies suitable for delivering Internet Protocol (IP) datagrams to connect their customers, such as dial-up, DSL, cable modems, fiber optic, wireless, or dedicated high-speed interconnects. Communication providers may also offer messaging services such as email, instant messaging, and SMS. Another type of communication provider is a Network Service Provider (NSP) that sells bandwidth or network access by providing direct backbone access to the Internet. Network Service Providers can include telecommunications companies, data carriers, wireless communication providers, Internet Service Providers, cable television operators that provide high-speed Internet access, etc.

[0108] The data switch 788 interconnects several modules within the ISP 770 and connects these modules to the user 782 via network 786. The data switch 788 can cover a small area where all the modules of the ISP 770 are in close proximity, or a large geographical area when the different modules are geographically dispersed. For example, the data switch 788 may include Fast Gigabit Ethernet (or even faster Gigabit Ethernet) within a data center rack, or an intercontinental virtual area network (VLAN).

[0109] User 782 uses client device 784 to access remote services. This client device includes at least a CPU, memory, display, and I / O. The client device can be a PC, mobile phone, netbook, tablet computer, gaming system, PDA, etc. In one implementation, ISP 770 identifies the type of device used by the client and adjusts the communication method accordingly. In other cases, the client device uses standard communication methods (such as HTML) to access ISP 770.

[0110] The embodiments of this disclosure can be practiced with various computer system configurations, including handheld devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, and the like. This disclosure can also be practiced in distributed computing environments, where tasks are performed by remote processing devices via wired or wireless network links.

[0111] In light of the above embodiments, it should be understood that this disclosure can employ various computer-implemented operations involving data stored in a computer system. These operations are those that require the physical manipulation of physical quantities. Any operation described herein that forms part of this disclosure is a useful machine operation. This disclosure also relates to means or apparatus for performing these operations. The apparatus may be specifically constructed for the desired purpose, or the apparatus may be a general-purpose computer selectively activated or configured by a computer program stored in a computer. Specifically, various general-purpose machines may be used with computer programs written in accordance with the teachings herein, or it may be more convenient to construct more specialized apparatus to perform the desired operations.

[0112] This disclosure can also be embodied as computer-readable code on a computer-readable medium. Alternatively, the computer-readable code can be downloaded from a server using the data exchange interconnect described above. A computer-readable medium is any data storage device capable of storing data that can subsequently be read by a computer system. Examples of computer-readable media include hard disk drives, network-attached storage devices (NAS), read-only memory, random access memory, CD-ROMs, CD-Rs, CD-RWs, magnetic tapes, and other optical and non-optical data storage devices. Computer-readable media may include computer-readable tangible media distributed across network-coupled computer systems, enabling the computer-readable code to be stored and executed in a distributed manner.

[0113] Although the method operations are described in a specific order, it should be understood that other housekeeping operations may be performed between operations, or operations may be adjusted so that they occur at slightly different times, or they may be distributed in a system that allows processing operations to occur at various intervals associated with the processing, as long as the processing of the overriding operations is performed in the desired manner.

[0114] While the foregoing disclosure has been described in considerable detail for the purposes of clarity, it will be apparent that certain changes and modifications may be practiced within the scope of the appended claims. Therefore, this embodiment is to be considered illustrative rather than restrictive, and this disclosure is not limited to the details given herein, but may be modified within the scope and equivalents of the described embodiments.

Claims

1. A computer-implemented method, the method comprising: Executing cloud video games in data centers; The video generated by the execution of the cloud video game is streamed to the client device via the network; Deploy cloud gaming agents to edge computing locations close to the client devices; The cloud gaming agent receives and buffers the video and retransmits lost packets of the video to the client device, wherein the retransmission of a given lost packet is in response to a request from the client device for the given lost packet. If the cloud gaming agent receives a request to retransmit lost packets at a rate exceeding a predefined threshold, the quality of the video is adjusted. as well as The adjusted video is streamed to the client device.

2. The method of claim 1, wherein streaming the video, buffering the video, and retransmitting lost packets are performed in real time.

3. The method of claim 1, wherein the retransmission of a given lost packet is in response to no acknowledgment of the given lost packet being received from the client device within a predefined time period following the previous transmission of the lost packet.

4. The method of claim 1, wherein the edge computing is located at a wireless base station that communicates wirelessly with the client device.

5. The method of claim 4, wherein the wireless base station is a member of a cellular network.

6. The method of claim 4, further comprising: A second cloud gaming agent is deployed to a second edge computing facility, which is located at a second wireless base station close to the wireless base station. The second cloud gaming agent receives and buffers the video. In response to detecting that the client device switches from the wireless base station to the second wireless base station, the second cloud gaming agent is then activated to retransmit the lost video packets to the client device.

7. The method of claim 6, wherein the buffering of the video by the second cloud gaming agent and the buffering of the video by the cloud gaming agent occur simultaneously before the client device switches from the wireless base station to the second wireless base station.

8. The method of claim 1, wherein the cloud gaming agent tracks video frames of the video and does not retransmit lost packets of the given video frame after a predefined time period has elapsed since the given video frame was transmitted to the client device, wherein the predefined time period is based on an expected time for rendering the video frame at the client device.

9. The method of claim 1, further comprising: The audio generated by the execution of the cloud video game is streamed to the client device via the network. The cloud gaming agent receives and buffers the audio and retransmits any lost audio packets to the client device.

10. A non-transitory computer-readable medium having program instructions that, when executed by a computing device, cause the computing device to perform a method comprising the following operations: Executing cloud video games in data centers; The video generated by the execution of the cloud video game is streamed to the client device via the network; Deploy cloud gaming agents to edge computing locations close to the client devices; The cloud gaming agent receives and buffers the video and retransmits lost packets of the video to the client device, wherein the retransmission of a given lost packet is in response to a request from the client device for the given lost packet. If the cloud gaming agent receives a request to retransmit lost packets at a rate exceeding a predefined threshold, the quality of the video is adjusted. as well as The adjusted video is streamed to the client device.

11. The non-transitory computer-readable medium of claim 10, wherein streaming the video, buffering the video, and retransmitting lost packets are performed in real time.

12. The non-transitory computer-readable medium of claim 10, wherein the retransmission of a given lost packet is in response to no acknowledgment of the given lost packet being received from the client device within a predefined time period following the previous transmission of the lost packet.

13. The non-transitory computer-readable medium of claim 10, wherein the edge computing is located at a wireless base station that wirelessly communicates with the client device.

14. The non-transitory computer-readable medium of claim 13, wherein the wireless base station is a member of a cellular network.

15. The non-transitory computer-readable medium of claim 13, wherein the non-transitory computer-readable medium further comprises: A second cloud gaming agent is deployed to a second edge computing facility, which is located at a second wireless base station close to the wireless base station. The second cloud gaming agent receives and buffers the video. In response to detecting that the client device switches from the wireless base station to the second wireless base station, the second cloud gaming agent is then activated to retransmit the lost video packets to the client device.

16. The non-transitory computer-readable medium of claim 15, wherein the buffering of the video by the second cloud gaming agent and the buffering of the video by the cloud gaming agent occur simultaneously before the client device switches from the wireless base station to the second wireless base station.

17. The non-transitory computer-readable medium of claim 10, wherein the cloud gaming agent tracks video frames of the video and does not retransmit lost packets of the given video frame after a predefined time period has elapsed since the given video frame was transmitted to the client device, wherein the predefined time period is based on an expected time for rendering the video frame at the client device.

18. The non-transitory computer-readable medium of claim 10, wherein the method further comprises: The audio generated by the execution of the cloud video game is streamed to the client device via the network. The cloud gaming agent receives and buffers the audio and retransmits any lost audio packets to the client device.

19. A computer-implemented system, the system comprising: A data center configured to execute cloud video games and stream video generated by the executed cloud video games to client devices via a network; as well as Edge computing, located close to the client device, includes a cloud gaming agent configured to receive and buffer the video and retransmit lost packets of the video to the client device. The cloud gaming agent is configured as follows: In response to a request from the client device for a given lost packet, the given lost packet is retransmitted; In response to receiving a request to retransmit lost packets at a rate exceeding a predefined threshold, the quality of the video is adjusted; and The adjusted video is streamed to the client device.

Images