A cloud game adaptive remote calling method and system based on SDK

Abstract

A kind of SDK-based cloud game adaptive remote calling method, comprising the following steps: cloud game platform client sends running cloud game request to SDK module by cloud game platform server;The SDK module sends running cloud game request to game server by game client;The game server utilizes adaptive push strategy, and game section is pushed to the game client;The game client utilizes bandwidth change perception algorithm to carry out the prediction of network bandwidth, and the HEVC video encoding algorithm based on visual characteristics is handled to cloud game image;The cloud game platform client receives cloud game image, and runs cloud game.The present application also provides a kind of SDK-based cloud game adaptive remote calling system, realizes that cloud game is switched smoothly and fluctuation is less, and bandwidth utilization is high, and overall user experience is improved.

Description

Technical Field

[0001] This invention relates to the field of image transmission technology, and in particular to an SDK-based cloud gaming adaptive remote invocation method and system. Background Technology

[0002] Communication technology is constantly innovating, with download speeds skyrocketing. From 3G to 4G and now to 5G, 5G is 10 to 100 times faster than 4G. This isn't just a simple explosion in transmission speed; it profoundly impacts every field, bringing tremendous changes to all industries. The widespread adoption of 5G will bring enormous commercial value to the communications sector and various vertical industries, including the gaming industry. In October 2018, Ubisoft Entertainment released the highly anticipated Nintendo Switch version of Assassin's Creed Odyssey, surprising many gamers. However, due to the Switch's performance limitations, running AAA titles on the console was impossible. But when people saw the "cloud gaming version" suffix on the poster, they realized that the cloud gaming era had begun.

[0003] Cloud gaming's ability to be played instantly without downloading, and its lack of client configuration requirements, has made it popular among users, a key reason for its widespread adoption. However, the biggest challenge for cloud gaming isn't the capabilities of the console itself, but rather the transmission network. 5G's high bandwidth, low latency, and high speed eliminate this problem. The gaming industry widely believes that cloud gaming is the future of gaming; after just one more generation of consoles, all games will become cloud games. Currently, cloud gaming is in its early stages of development, plagued by unstable networks, imperfect push services, and significant latency in user requests and server responses, severely impacting user experience and hindering its widespread adoption. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the present invention aims to provide an SDK-based adaptive remote invocation method and system for cloud gaming. Based on the HTTP / 2 SDK, it employs an adaptive push strategy, a client bandwidth change perception algorithm, and a visually-based HEVC (High Efficiency Video Coding) video encoding algorithm to achieve smooth cloud gaming switching with minimal fluctuations, high bandwidth utilization, and an overall improved user experience.

[0005] To achieve the above objectives, the present invention provides an SDK-based cloud gaming adaptive remote invocation method, comprising the following steps:

[0006] The cloud gaming platform client sends the request to run a cloud game to the SDK module through the cloud gaming platform server;

[0007] The SDK module sends a request to run a cloud game to the game server through the game client;

[0008] The game server uses an adaptive push strategy to push game segments to the game client;

[0009] The game client uses a bandwidth change perception algorithm to predict network bandwidth and a visual characteristic-based HEVC video coding algorithm to process cloud game images.

[0010] The cloud gaming platform client receives cloud gaming images and runs cloud games.

[0011] Furthermore, the adaptive push strategy includes the following: the HTTP2 game server does not need to wait for each cloud game segment request to arrive before responding, and pushes the required and ready game segment to the cloud gaming platform client after receiving the first request.

[0012] Furthermore, the step of predicting network bandwidth using the bandwidth change sensing algorithm also includes bandwidth estimation based on packet loss: when the detected packet loss rate is greater than a preset value, the network is considered congested, and the sending speed is reduced; when the detected packet loss rate is less than the preset value, the network is considered unobstructed, and the sending speed is increased.

[0013] Furthermore, the step of predicting network bandwidth using the bandwidth change sensing algorithm further includes delay estimation:

[0014] Receive cloud gaming segments and extract measurement information from the previous push cycle of the cloud gaming segment, using the average value as a statistical result to represent the possible future network bandwidth;

[0015] The statistical results are used to estimate the delay.

[0016] When the game client detects network bandwidth congestion, it reduces the number of push notifications or lowers the bitrate of cloud gaming segments;

[0017] If multiple push cycles with bandwidth predictions indicating delays occur consecutively, a control frame will be generated directly using flow control features to terminate the current stream's transmission. Simultaneously, the game server will be advised to revert to a previous transmission state.

[0018] Furthermore, the step of processing cloud gaming images using the HEVC video coding algorithm based on visual characteristics also includes:

[0019] Based on the texture complexity and temporal motion intensity of the image region, the spatiotemporal content-aware factor is designed as the largest coding unit, which effectively improves the subjective visual quality of the encoded video while keeping the output bitrate almost unchanged. Large weights are assigned to visually important image regions, and small weights are assigned to visually weak image regions, thereby improving the performance of the encoder.

[0020] To achieve the above objectives, the present invention also provides a cloud gaming adaptive remote invocation system based on an SDK, comprising a cloud gaming platform client, a cloud gaming platform server, an SDK module, a game client, and a game server, wherein...

[0021] The cloud gaming platform client sends a request to run a cloud game to the cloud gaming platform server and receives a response from the cloud gaming platform server to run the cloud game.

[0022] The cloud gaming platform server receives cloud gaming platform client requests to run cloud games and processes the game screen.

[0023] The SDK module provides the cloud gaming platform with an adaptive push strategy, a bandwidth change awareness algorithm, and a visual characteristic-based HEVC video encoding algorithm to achieve smooth operation of cloud games.

[0024] The game client performs bandwidth estimation and latency estimation based on the bandwidth change perception algorithm of the SDK module.

[0025] The game server provides game segments to the cloud gaming platform client based on the adaptive push strategy of the SDK module.

[0026] Furthermore, the SDK module, based on HTTP / 2.0 features, provides an adaptive push strategy for the game server, which pushes the required and ready game segments to the game client after receiving the first request.

[0027] Furthermore, the SDK module, based on HTTP / 2.0 features, provides the game client with a bandwidth change awareness algorithm to perform bandwidth estimation and latency estimation.

[0028] Furthermore, the SDK module designs the spatiotemporal content-aware factor as the maximum encoding unit based on the texture complexity and temporal motion intensity of the image region, effectively improving the subjective visual quality of the encoded video while keeping the output bitrate almost unchanged; it assigns greater weight to visually important image regions and less weight to visually weak image regions, thereby improving the encoder's performance.

[0029] Furthermore, the SDK module is integrated into the cloud gaming platform of the game client.

[0030] The SDK-based cloud gaming adaptive remote invocation method and system of the present invention has the following advantages compared with the prior art:

[0031] Leveraging the new features of HTTP / 2.0, such as binary framing, server push, multiplexing, and encoding, this approach solves problems related to cloud gaming startup latency, request-response latency, reduced video jitter, network instability, and imperfect push services. Considering the impact of client-side cache startup thresholds and network fluctuations, the size of the receiving-end cache is controlled within an optimal range by sensing startup thresholds and network bandwidth. This results in smooth cloud gaming switching with minimal fluctuations, high bandwidth utilization, and an overall improved user experience.

[0032] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. Attached Figure Description

[0033] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0034] Figure 1 The flowchart below illustrates the SDK-based cloud gaming adaptive remote invocation method according to the present invention.

[0035] Figure 2 This is a schematic diagram of the SDK-based cloud gaming adaptive remote invocation system according to the present invention;

[0036] Figure 3 This is a schematic diagram of the cloud gaming push service according to the present invention;

[0037] Figure 4 This is a schematic diagram of cloud gaming latency estimation according to the present invention. Detailed Implementation

[0038] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0039] Example 1

[0040] Figure 1 The flowchart of the SDK-based cloud gaming adaptive remote invocation method according to the present invention will be referenced below. Figure 1 The present invention provides a detailed description of the SDK-based cloud gaming adaptive remote invocation method.

[0041] First, in step 101, a request to run a cloud game is sent.

[0042] In this embodiment of the invention, the cloud gaming platform client sends a request to run a cloud game to the cloud gaming platform server and receives a response from the cloud gaming platform server.

[0043] In step 102, the request to run the cloud game is forwarded to the SDK module.

[0044] In this embodiment of the invention, the cloud gaming platform server forwards the request to run a cloud game to the SDK module and receives the response from the SDK module.

[0045] In step 103, the SDK module sends a cloud gaming request to the game client.

[0046] In this embodiment of the invention, the SDK module utilizes the new features of HTTP / 2.0, such as binary framing, server push, multiplexing, and encoding, to design a new push scheme for cloud gaming transmission. This aims to solve problems such as cloud game startup latency, request-response latency, reduced video jitter, network instability, and imperfect push services. Based on the new features of this protocol, and considering the impact of client-side cache startup thresholds and network fluctuations, a dynamic adaptive algorithm based on cloud game segment push is proposed. By sensing the startup threshold and network bandwidth, the size of the receiving end's buffer is controlled within an optimal range, achieving smooth cloud game switching with minimal fluctuations, high bandwidth utilization, and an overall improved user experience.

[0047] In this embodiment of the invention, the SDK module provides a server adaptive push strategy, a client bandwidth change awareness algorithm, and a visual characteristic-based HEVC (High Efficiency Video Coding) video coding algorithm.

[0048] Based on the server's adaptive push strategy, the HTTP / 2 game server does not need to wait for each cloud game segment request to arrive before responding. Instead, it proactively pushes the game segments that will be needed in the future and are ready to be played to the client after receiving the first request. This allows the client to minimize the number of requests, and in extreme cases, may only need to exchange the stream once. Figure 3 As shown.

[0049] Client-side bandwidth change sensing algorithms predict network bandwidth changes using two methods:

[0050] One method of bandwidth estimation is based on packet loss. More packet loss indicates greater network congestion, and the sending speed is reduced to alleviate this congestion. If there is no packet loss, the network is considered to be in good condition, and the sending speed can be increased. A packet loss rate greater than 1% is considered high. Another method is based on latency estimation, such as... Figure 4 As shown, this method requires using local measurement statistics for prediction. This necessitates measuring previously received cloud gaming segments on the client side and extracting measurement information from the previous push cycle. The average value is then used as the statistical result to represent the potential future network bandwidth. When the client perceives a deterioration in network bandwidth, the push service reduces the number of pushes or lowers the bitrate of the cloud gaming segments. If multiple push cycles show delayed bandwidth predictions consecutively, the client needs to directly send a control frame through flow control features to terminate the current stream's transmission, while simultaneously suggesting that the server revert to a previous transmission state.

[0051] The HEVC video coding algorithm, based on visual characteristics, incorporates rate-distortion optimization techniques across intra-frame and inter-frame prediction coding modes within the HEVC standard. It fully considers image content features and human visual characteristics to achieve visually optimal coding mode selection. A spatiotemporal content-aware factor is designed based on the texture complexity and temporal motion intensity of image regions, introducing new bitrate allocation weights for the largest coding unit (CTU). This results in more rational bitrate allocation, effectively improving the subjective visual quality of the encoded video while maintaining a nearly constant output bitrate. The core of this video coding algorithm is to allocate greater weight (sufficient bits) to visually important image regions and less weight to visually weak image regions. This algorithm allows coding control technology to maximize encoder performance with limited bandwidth resources. The application of this technology enables cloud gaming to achieve the same encoding quality at a lower bitrate, and the resulting bitrate savings effectively reduce operating costs.

[0052] In this embodiment of the invention, the HEVC video coding algorithm based on visual characteristics uses 32x32 pixels as the basic unit. This choice of large-size transform unit allows the encoder to significantly improve the compression rate when processing high-resolution images. At the same time, in order to cope with the diversity of textures in high-definition videos, the algorithm provides 35 prediction modes for luminance signals, including 33 angle prediction modes, DC prediction mode, and Plannar prediction mode, thereby better matching complex textures in the video, improving prediction results, and removing redundant space.

[0053] In this embodiment of the invention, the SDK module uses the HEVC video encoding and decoding algorithm to reduce motion parameter redundancy between adjacent coding units by utilizing spatial and temporal correlations, thereby improving quality, reducing capacity, and saving bandwidth, effectively improving the response speed of the game server.

[0054] In step 104, the game client sends a request to run the cloud game to the game server and receives a response from the game server.

[0055] In step 105, the game server responds to the cloud gaming request and pushes the game segment to the cloud gaming platform client.

[0056] Example 2

[0057] The present invention also provides an SDK-based cloud gaming adaptive remote invocation system, which utilizes an adaptive push strategy, a client bandwidth change perception algorithm, and a visual characteristic-based HEVC video encoding algorithm to perform adaptive remote invocation of cloud games.

[0058] Figure 2 This is a schematic diagram of the SDK-based cloud gaming adaptive remote invocation system architecture according to the present invention, as shown below. Figure 2 As shown, the SDK-based cloud gaming adaptive remote invocation system of the present invention includes a cloud gaming platform client 10, a cloud gaming platform server 20, an SDK module 30, a game client 40, and a game server 50, wherein...

[0059] The cloud gaming platform client 10 sends a request to run a cloud game to the cloud gaming platform server 20, receives the response from the cloud gaming platform server 20, and runs the cloud game.

[0060] The cloud gaming platform server 20 receives cloud gaming platform client 10 requests to run cloud games and processes the game screen.

[0061] SDK module 30 leverages the binary framing, server push, multiplexing, and encoding features of HTTP / 2.0 to provide server adaptive push strategies, client bandwidth change awareness algorithms, and HEVC video encoding algorithms based on visual characteristics. This enables smooth cloud gaming switching with minimal fluctuations, high bandwidth utilization, and an overall improved user experience.

[0062] In this embodiment of the invention, the server adaptive push strategy means that the HTTP2 game server does not need to wait for each cloud game segment request to arrive before responding. Instead, it proactively pushes the game segments that will be needed in the future and are ready to be prepared to the client after receiving the first request.

[0063] Client-side bandwidth change sensing algorithms predict network bandwidth changes using two methods:

[0064] One method of bandwidth estimation is based on packet loss. More packet loss indicates greater network congestion, and the sending speed is reduced to alleviate this congestion. If there is no packet loss, the network is considered to be in good condition, and the sending speed can be increased. A packet loss rate greater than 1% is considered high. Another method is based on latency estimation, such as... Figure 4As shown, this method requires using local measurement statistics for prediction. This necessitates measuring previously received cloud gaming segments on the client side and extracting measurement information from the previous push cycle. The average value is then used as the statistical result to represent the potential future network bandwidth. When the client perceives a deterioration in network bandwidth, the push service reduces the number of pushes or lowers the bitrate of the cloud gaming segments. If multiple push cycles show delayed bandwidth predictions consecutively, the client needs to directly send a control frame through flow control features to terminate the current stream's transmission, while simultaneously suggesting that the server revert to a previous transmission state.

[0065] The HEVC video coding algorithm, based on visual characteristics, incorporates rate-distortion optimization techniques across intra-frame and inter-frame prediction coding modes within the HEVC standard. It fully considers image content features and human visual characteristics to achieve visually optimal coding mode selection. A spatiotemporal content-aware factor is designed based on the texture complexity and temporal motion intensity of image regions, introducing new bitrate allocation weights for the largest coding unit (CTU). This results in more rational bitrate allocation, effectively improving the subjective visual quality of the encoded video while maintaining a nearly constant output bitrate. The core of this video coding algorithm is to allocate greater weight (sufficient bits) to visually important image regions and less weight to visually weak image regions. This algorithm allows coding control technology to maximize encoder performance with limited bandwidth resources. The application of this technology enables cloud gaming to achieve the same encoding quality at a lower bitrate, and the resulting bitrate savings effectively reduce operating costs.

[0066] In this embodiment of the invention, the SDK module uses the HEVC video encoding and decoding algorithm to reduce motion parameter redundancy between adjacent coding units by utilizing spatial and temporal correlations, thereby improving quality, reducing capacity, and saving bandwidth, effectively improving the response speed of the game server.

[0067] The game client 40 receives cloud game requests from the SDK module and uses a bandwidth change perception algorithm and a visual-based HEVC video coding algorithm to predict network bandwidth changes and perform video processing.

[0068] The game server 50 receives cloud game requests from the game client 40 and pushes game segments to the game client using an adaptive push strategy.

[0069] The present invention provides a cloud gaming adaptive remote invocation method and system based on an SDK. It fully utilizes the new features of HTTP / 2 and integrates technologies such as binary framing, server push, multiplexing, and encoding into an SDK. The SDK mainly uses server adaptive push strategy, client bandwidth change perception algorithm, and HEVC video encoding algorithm based on visual characteristics to enable the SDK to have adaptive remote invocation function.

[0070] This SDK is developed for cloud gaming platform clients and integrated into every cloud game running on that platform. The SDK utilizes the HEVC video encoding / decoding algorithm, which leverages spatial and temporal correlations to reduce motion parameter redundancy between adjacent coding units, improving quality, reducing file size, and saving bandwidth, effectively enhancing the game server's response speed.

[0071] It will be understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An SDK-based cloud game adaptive remote calling method, characterized in that, The method comprises the following steps: The cloud game platform client sends a cloud game running request to the SDK module through the cloud game platform server; The SDK module sends the cloud game running request to the game server through the game client; The game server uses a game server adaptive push strategy based on HTTP2 to push the game segment to the game client; The game client uses a bandwidth change perception algorithm to predict the network bandwidth, which includes a bandwidth estimation based on packet loss rate or a delay estimation, and dynamically adjusts the transmission strategy according to the prediction result; Meanwhile, the HEVC video coding algorithm based on visual characteristics designs a space-time content perception factor according to the texture complexity of the image region and the time domain motion intensity, allocates a code rate weight for the maximum coding unit, and processes the cloud game image; The cloud game platform client receives the cloud game image and runs the cloud game; The game server adaptive push strategy based on HTTP2 includes that the HTTP2 game server does not need to respond when each cloud game segment request arrives, and pushes the game segment that needs to be pushed and is ready to the cloud game platform client after receiving the first request.

2. The SDK-based cloud game adaptive remote invocation method according to claim 1, characterized in that, The bandwidth estimation based on packet loss includes that when the packet loss rate is detected to be greater than a preset value, it is considered that the network is congested, and the sending speed is reduced at this time; when the packet loss rate is detected to be less than a preset value, it is considered that the network is smooth, and the sending speed is increased at this time.

3. The SDK-based cloud game adaptive remote invocation method according to claim 1, characterized in that, The delay estimation includes: Receiving a cloud game segment and extracting the measurement information of a push cycle before the cloud game segment, using the average value as the statistical result to represent the possible network bandwidth in the future; Using the statistical result to estimate the delay; When the game client perceives that the network bandwidth is blocked, the push quantity is reduced or the bit rate of the cloud game segment is reduced; If a plurality of push cycles with a delay prediction result appear continuously, a control frame is directly generated to terminate the transmission of the current flow through the flow control characteristics, and the game server is recommended to return to a certain transmission state.

4. The SDK-based cloud game adaptive remote invocation method according to claim 1, characterized in that, The HEVC video coding algorithm based on visual characteristics for processing the cloud game image further comprises, In the image region with visual importance, a large weight is allocated, and in the image region with weak visual importance, a small weight is allocated, to improve the performance of the encoder.

5. An SDK-based cloud game adaptive remote invocation system, adopting the method of any one of claims 1-4, characterized in that, The method comprises a cloud game platform client, a cloud game platform server, an SDK module, a game client, and a game server, wherein The cloud game platform client sends a cloud game running request to the cloud game platform server and receives a return response from the cloud game platform server to run the cloud game; The cloud game platform server receives the cloud game running request from the cloud game platform client and processes the game picture; The SDK module provides an adaptive push strategy, a bandwidth change perception algorithm, and an HEVC video coding algorithm based on visual characteristics for the cloud game platform to achieve smooth running of the cloud game; The game client estimates the bandwidth and the delay based on the bandwidth change perception algorithm of the SDK module, The game server provides game segments for the cloud game platform client based on the adaptive pushing strategy of the SDK module.

6. The SDK-based cloud game adaptive remote calling system according to claim 5, characterized in that, The SDK module provides an adaptive pushing strategy for the game server based on the HTTP2.0 feature, and the game server pushes the game segments that need to be pushed and are ready after receiving the first request to the game client.

7. The SDK-based cloud game adaptive remote invocation system according to claim 5, wherein, The SDK module provides a bandwidth change perception algorithm for the game client based on the HTTP2.0 feature, and performs bandwidth estimation and delay estimation.

8. The SDK-based cloud game adaptive remote invocation system according to claim 5, wherein, The SDK module designs a space-time content perception factor according to the texture complexity and time domain motion intensity of the image region as the maximum coding unit, effectively improves the subjective visual quality of the encoded video while keeping the output code rate almost unchanged, and improves the performance of the encoder by allocating more weight to the visually important image region and less weight to the visually weak image region.

9. The SDK-based cloud game adaptive remote invocation system according to claim 5, wherein, The SDK module is integrated on the cloud game of the game client.

Images