Streaming transmission and low-delay decoding method based on cloud game

Abstract

The invention relates to the technical field of computers and data processing, discloses a streaming transmission and low-delay decoding method based on a cloud game, and aims to solve the problems of high transmission delay, low coding and decoding efficiency and split interaction response in the prior art. The method comprises the following steps: extracting dynamic characteristics of a game picture, dynamically adjusting a coding rate and a resolution ratio through a reinforcement learning model based on network state parameters and the dynamic characteristics, generating a current frame prediction result by using an AI prediction model, and preferentially transmitting a key frame patch, decoding is accelerated through a special hardware decoding chip, asynchronous rapid decoding is adopted for key frame patches, and frames and user input events are decoded synchronously to control the end-to-end total delay to be smaller than or equal to 50 ms. According to the technical scheme, the loss rate and the average transmission delay of the key picture area can be remarkably reduced, the decoding delay is reduced, it is ensured that the total delay is stabilized within 50 ms, and the interaction synchronism and the operation response accuracy of the high-speed dynamic game are improved.

Description

Technical Field

[0001] This invention belongs to the field of computer and data processing technology, specifically relating to a streaming and low-latency decoding method based on cloud gaming. Background Technology

[0002] With the deep integration of cloud computing and 5G communication technologies, cloud gaming, as a new form of entertainment that migrates high-performance computing tasks to the cloud and enables real-time interaction through video streaming, is gradually becoming an important development direction for digital content services. The core of cloud gaming lies in running game logic on a remote server, which, after efficient encoding and compression, transmits it to the terminal device as a low-latency video stream for decoding and rendering, thus achieving a high-quality gaming experience without requiring local high-performance hardware. This model relies on high-throughput, low-latency network transmission links and efficient encoding and decoding processing capabilities, and its technical system covers multiple key aspects such as video encoding, network transmission, terminal decoding, and input feedback.

[0003] Cloud gaming's stringent end-to-end latency requirements distinguish it from traditional video streaming services. Existing technologies generally employ advanced encoding standards such as H.265 or AV1 combined with real-time transmission protocols like WebRTC, achieving some progress in compression efficiency and transmission reliability. However, they still haven't fully adapted to the unique dual characteristics of cloud gaming: "high dynamic range visuals + strong real-time interaction." Specifically, traditional streaming strategies are mostly based on fixed or static variable bitrate mechanisms, making it difficult to dynamically adjust according to network bandwidth fluctuations and the complexity of the visual content. The encoding process lacks targeted optimization for high-frequency moving objects in the game scene (such as character movement and particle effects), leading to increased redundant data or loss of key details. If terminal decoding relies on general-purpose CPU software decoding, latency is significantly high, while dedicated hardware decoding, although faster, faces problems such as poor chip compatibility and difficulty in power consumption control. More importantly, user operation commands must go through a complete closed loop of terminal upload, cloud processing, re-encoding, and transmission. The added transmission and encoding / decoding latency easily causes a disconnect between operation response and visual feedback, severely impacting the immersion and competitive fairness of time-sensitive games such as shooting and racing.

[0004] Existing technical solutions optimize each stage in isolation, lacking an end-to-end collaborative mechanism from cloud encoding strategies and network transmission scheduling to terminal decoding execution. Especially in dynamic network environments, they cannot guarantee the priority delivery and rapid rendering of keyframes, nor do they effectively integrate AI prediction and hardware acceleration capabilities to compress processing latency. Therefore, there is an urgent need for a low-latency streaming method that deeply integrates dynamic bitrate adaptation, intelligent predictive encoding, and hardware collaborative decoding. This method should systematically solve the interactive fragmentation problems caused by transmission jitter, inefficient encoding, and decoding lag in cloud gaming, achieving a technological breakthrough by stably controlling end-to-end latency to within 50 milliseconds. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a cloud gaming-based streaming and low-latency decoding method, which can effectively solve the problems in the background technology. To achieve the above objective, this invention provides the following technical solution: a cloud gaming-based streaming and low-latency decoding method, which includes the following steps: Step S1, extracting dynamic features of the game screen, including motion vector density, color change rate, and key object trajectory; Step S2, based on network state parameters and dynamic features, dynamically adjusting the encoding bitrate and resolution through a reinforcement learning model, supporting layered encoding; Step S3, using an AI prediction model to generate the prediction result of the current frame, combining intra-frame / inter-frame prediction encoding, marking areas with large prediction errors as keyframe patches and prioritizing their transmission; Step S4, accelerating decoding through a dedicated hardware decoding chip, using asynchronous fast decoding for keyframe patches; Step S5, synchronously decoding frames and user input events, with a total end-to-end latency of ≤50ms.

[0006] Preferably, the extraction process of dynamic features of the game screen in step S1 is as follows: the cloud server obtains the game screen sequence with a frame rate of not less than 60fps in real time through the game engine application interface; performs motion estimation calculation on the continuous frame sequence, generates a motion vector field and counts the proportion of pixels with motion vector amplitude exceeding a preset threshold within a unit area as the motion vector density; calculates the root mean square error of the color components of each pixel between adjacent frames as the color change rate; identifies key objects of a preset category in the screen based on the target detection algorithm, and tracks the continuous change of their center coordinates to form a trajectory sequence.

[0007] Furthermore, in step S2, the state space of the reinforcement learning model is defined as a three-dimensional vector [motion vector density, color change rate, key object trajectory curvature], and the action space includes a set of bitrate adjustment instructions {+20%, +10%, 0%, -10%, -20%} and a set of resolution switching instructions {4K, 1440p, 1080p}; the reward function is designed as R=α×(50-T_total)+β×PSNR-γ×PLR, where T_total is the measured end-to-end latency, PSNR is the peak signal-to-noise ratio, PLR is the packet loss rate, and α, β, and γ are weight coefficients that satisfy α>β>γ>0.

[0008] Furthermore, in step S2, the Base layer of the layered encoding uses a fixed quantization parameter of QP=32, which includes low-frequency DCT coefficients and basic motion vector information; the Enhancement layer uses a variable quantization parameter of QP=24, which supplements high-frequency detail coefficients and fine motion vectors; the Base layer data packets are marked with a priority of DSCP=46, and the Enhancement layer data packets are marked with a priority of DSCP=26, thus ensuring transmission reliability through a differentiated quality of service strategy.

[0009] Preferably, in step S3, the lightweight convolutional neural network adopts an encoder-decoder architecture. The encoder contains 3 convolutional layers and 2 max pooling layers, and the decoder contains 2 deconvolutional layers and 1 output convolutional layer. The network input is 5 consecutive frames of 128×128 pixel YUV420 format image blocks, and the output is the prediction result of the corresponding image block in the current frame. The training loss function is L=0.7×L_MSE+0.3×L_SSIM, where L_MSE is the mean squared error and L_SSIM is the structural similarity index.

[0010] Furthermore, in step S3, intra-frame prediction employs a full-search algorithm with 35 prediction modes, forcing the use of DC mode or planar mode for regions marked as keyframe patches to reduce computational complexity; inter-frame prediction employs an improved diamond search algorithm, limiting the search range to a 16×16 pixel area; the marking condition for keyframe patches is that the prediction error exceeds a threshold. , where μ is the mean of the global prediction error and σ is the standard deviation.

[0011] In addition, the dedicated hardware decoding chip in step S4 integrates a programmable pipeline architecture, including an entropy decoding unit, an inverse quantization unit, an inverse transform unit, and a motion compensation unit; the motion compensation unit has 8 built-in parallel interpolation filters, supporting bilinear interpolation calculation with 1 / 4 pixel precision; the asynchronous fast decoding mechanism allocates an independent decoding channel for key frame patches, and starts patch frame decoding when the basic frame decoding completion rate reaches 80%, and the two are fused at the pixel level in the frame buffer area.

[0012] Preferably, in step S5, the timestamp synchronization mechanism uses a precision time protocol to ensure that the clock deviation between the cloud and the terminal is ≤1ms; the user input event is encapsulated as a 96-byte binary protocol data unit, including an 8-byte timestamp, a 4-byte operation type identifier, and 84 bytes of operation parameters; the binding relationship between the input event and the video frame is established through a 64-bit sequence number to ensure that the logical processing delay does not exceed 20ms.

[0013] Furthermore, the end-to-end delay control strategy includes a dual-loop system of feedforward control and feedback control: feedforward control dynamically adjusts the coding complexity based on the network state prediction model, and disables B-frame encoding when RTT > 80ms; feedback control is based on the delay observer output, and the terminal enables the motion-compensated frame interpolation algorithm when T_total > 60ms, with interpolation weight coefficients... .

[0014] Compared with existing technologies, this invention has the following advantages: 1. By co-optimizing AI predictive coding and layered transmission, the loss rate of critical screen areas under network fluctuations is reduced by 40%, and the average transmission latency is compressed from 80ms to 35ms; 2. A dedicated hardware decoding chip combined with an asynchronous decoding strategy reduces the decoding latency from 50ms in traditional software decoding to 15ms, and is compatible with mainstream general-purpose graphics processors and application-specific integrated circuit architectures; 3. An end-to-end latency co-control mechanism ensures that the total latency is kept stable within 50ms, achieving the standard of smooth local operation on the host, and significantly improving the interactive synchronization and operation response accuracy of high-speed dynamic games. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the overall technical solution architecture of the cloud gaming-based streaming and low-latency decoding method proposed in this invention; Figure 2 This is a schematic diagram of the core principle framework of the dynamic bitrate and resolution adjustment mechanism based on reinforcement learning in this invention. Figure 3 This is a flowchart illustrating the logical flow of AI predictive coding and keyframe patch priority transmission in this invention. Detailed Implementation To further illustrate the technical means and effects adopted by the present invention to achieve the intended purpose, the specific embodiments according to the present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments.

[0016] Example 1 In large-scale multiplayer online competitive games, cloud servers acquire game footage sequences with a stable frame rate of 60 frames per second in real time via the game engine's application programming interface (API). These sequences include fast-moving character skill effects and complex scene transitions. See also... Figure 1 The system architecture includes a dynamic feature extraction module, an adaptive encoding control module, a predictive encoding optimization module, a hardware-accelerated decoding module, and a latency synchronization control module. These modules work together to achieve the goal of streaming transmission with an end-to-end latency of no more than 50 milliseconds.

[0017] The dynamic feature extraction module performs the operation described in step S1, performing motion estimation calculations on the continuous frame sequence to generate a motion vector field containing horizontal and vertical displacement components. The percentage of pixels with motion vector amplitudes exceeding 8 pixels per unit area is used as the motion vector density parameter, with a value ranging from 0 to 1 and a precision of three decimal places. The root mean square error of the luminance and chrominance components in the YUV color space of each pixel between adjacent frames is calculated as the color change rate, and the statistical value is updated every 5 frames using a sliding window mechanism. Based on the YOLOv5 object detection algorithm, three key objects—preset player characters, skill effects, and environmental objects—are identified in the scene. A Kalman filter tracks the continuous changes in their center coordinates, forming a trajectory sequence containing position coordinates, motion speed, and direction angle. The trajectory data is updated at a frequency of 30 sampling points per frame.

[0018] The adaptive coding control module performs the operation described in step S2, see [link / reference]. Figure 2 The state space of the reinforcement learning model is defined as a three-dimensional vector, containing motion vector density, color change rate, and key object trajectory curvature. The trajectory curvature is obtained by calculating the rate of change of the tangent angle over five consecutive trajectory points. The action space contains five bitrate adjustment instructions and three resolution switching instructions. The bitrate adjustment instructions correspond to ±20%, ±10%, and the original bitrate, respectively. The resolution switching instructions correspond to three resolutions: 3840×2160, 2560×1440, and 1920×1080. The measured end-to-end latency in the reward function is obtained using the timestamp interpolation method. The peak signal-to-noise ratio is calculated using the standard 8-bit grayscale image calculation method. The packet loss rate is statistically analyzed over a period of 100 milliseconds. The weight coefficients are set to α=0.6, β=0.3, and γ=0.1. The model performs a policy update every 200 milliseconds.

[0019] In the layered coding mechanism, the Base layer uses a fixed quantization parameter of 32 for encoding, including the DC component and the discrete cosine transform coefficients of the first three AC components, as well as basic motion vector information at the 16×16 pixel block level. The packet distinguishing service code point is marked as 46, corresponding to the network accelerated forwarding service level. The Enhancement layer uses a variable quantization parameter of 24 for encoding, supplementing high-frequency detail coefficients and fine motion vectors at the 4×4 pixel block level. The distinguishing service code point is marked as 26, corresponding to the guaranteed forwarding service level. Base layer packets use forward error correction coding with a redundancy setting of 20%, while Enhancement layer packets use an automatic retransmission request mechanism with a maximum retransmission limit of 3 times.

[0020] The predictive coding optimization module performs the operation described in step S3, see [link / reference]. Figure 3The lightweight convolutional neural network employs an encoder-decoder architecture. The encoder consists of three convolutional layers and two max-pooling layers, with kernel sizes of 3×3, 5×5, and 3×3, and channel numbers of 32, 64, and 128 respectively. The max-pooling window size is 2×2. The decoder consists of two deconvolutional layers and one output convolutional layer. The deconvolutional kernel size is 4×4, with channel numbers of 64 and 32 respectively. The output convolutional layer uses a 1×1 kernel to generate prediction results. The network input consists of five consecutive frames of 128×128 pixel YUV420 format image patches with a batch size of 32. The Adam optimizer is used during training, with an initial learning rate of 0.001, which decays by a factor of 0.1 every 10 epochs.

[0021] Intra-frame prediction employs a full-search algorithm with 35 prediction modes, traversing all prediction directions across a 16×16 pixel macroblock and calculating the rate-distortion cost for each mode. For regions marked as keyframe patches, DC mode or planar mode is enforced to reduce computational complexity, and the macroblock partitioning is limited to a single 16×16 partition. Inter-frame prediction uses an improved diamond search algorithm, with the initial search point set as the motion vector prediction value. The search range is limited to a 16×16 pixel area, and the search template contains 9 checkpoints. The marking condition for keyframe patches is that the prediction error exceeds a dynamic threshold, which is calculated by adding twice the standard deviation to the global prediction error mean. The error is calculated using an absolute error summation algorithm, and the statistical window size is 100 macroblocks.

[0022] The hardware-accelerated decoding module executes the operation described in step S4. The dedicated hardware decoding chip integrates a programmable pipeline architecture, including an entropy decoding unit, an inverse quantization unit, an inverse transform unit, and a motion compensation unit. The entropy decoding unit supports both context-adaptive binary arithmetic coding and exponential Golomb coding formats, achieving a processing throughput of 5 gigabits per second. The inverse quantization unit uses 32-bit fixed-point arithmetic, with a reconstruction coefficient accuracy error not exceeding 0.5%. The inverse transform unit supports integer discrete cosine transforms of both 4×4 and 8×8 sizes, with a fixed operation cycle of 8 clock cycles. The motion compensation unit incorporates 8 parallel interpolation filters, supporting bilinear interpolation calculations with 1 / 4 pixel precision, and has a reference frame buffer capacity of 12 frames.

[0023] The asynchronous fast decoding mechanism allocates an independent decoding channel for keyframe patches. The patch frame decoding process is initiated immediately when the basic frame decoding completion rate reaches 80%. Basic frame decoding uses the main pipeline, while patch frame decoding uses a backup pipeline. The two are fused pixel-level in the frame buffer. The fusion process employs the alpha mixing algorithm, with a basic frame pixel weight of 0.7 and a patch frame pixel weight of 0.3. The fused frame data is then sent to the post-processing unit for deblocking filtering and color space conversion.

[0024] The delay synchronization control module executes the operation described in step S5. The timestamp synchronization mechanism uses a precise time protocol to synchronize the cloud and terminal clocks. The master clock period is 125 milliseconds, and the slave clock deviation compensation value is updated every 10 seconds. User input events are encapsulated as 96-byte binary protocol data units, including an 8-byte nanosecond-level timestamp, a 4-byte operation type identifier, and 84 bytes of operation parameters. The operation types cover 12 events such as button status, mouse movement, and gamepad vibration. The binding relationship between input events and video frames is established through 64-bit sequence numbers. The sequence number generator uses a monotonically increasing algorithm, which can assign 1000 unique sequence numbers per millisecond.

[0025] The end-to-end latency control strategy comprises a dual-loop system of feedforward control and feedback control. Feedforward control dynamically adjusts the coding complexity based on the network state prediction model. When the round-trip latency exceeds 80 milliseconds, bidirectional predictive frame coding is immediately disabled, and only intra-frame coding and forward predictive frames are used. Feedback control is based on the output of a latency observer with a sampling period of 50 milliseconds. When the total end-to-end latency exceeds 60 milliseconds, the terminal enables a motion-compensated frame interpolation algorithm. The interpolation weight coefficients are calculated using the difference between the current latency value and the target latency value, and the weight coefficients are limited to a range of 0 to 1. The time taken to generate the interpolated frame is controlled within 3 milliseconds.

[0026] Example 2 In mobile cloud gaming applications, the terminal device is equipped with a dedicated hardware decoding chip, the network environment is a 5G mobile communication network, and the game content is a first-person shooter. The dynamic feature extraction module, tailored to the characteristics of mobile screen size, divides the motion vector density statistical area into 9 grids and calculates the independent parameters for each grid. The color change rate calculation uses a simplified algorithm, only counting changes in the luminance component, and the sampling interval is adjusted to once every 10 frames. Key object recognition focuses on three elements: the player's crosshair, enemy characters, and ammunition supplies. The trajectory tracking algorithm uses a lightweight optical flow method, reducing computational resource consumption by 40%.

[0027] The adaptive coding control module optimizes the reinforcement learning model for mobile network characteristics. The state vector adds two dimensions: signal strength indication and carrier aggregation state. A 720p resolution option is added to the action space. The packet loss rate weight in the reward function is increased to 0.2, and the model update frequency is adjusted to once every 500 milliseconds. The quantization parameter of the base layer in the hierarchical coding is adjusted to 36, while the enhancement layer uses dynamic quantization parameters, adjusted between 22 and 28 based on network throughput. The forward error correction redundancy of the base layer is set to 30%, and the maximum retransmission count of the enhancement layer is reduced to 2.

[0028] The input size of the convolutional neural network in the predictive coding optimization module has been reduced to 64×64 pixels, and the number of channels in the convolutional layer has been halved accordingly. Intra-frame prediction modes have been reduced to 9 fast modes, and the inter-frame search range is limited to an 8×8 pixel area. The keyframe patch marker threshold calculation formula has been adjusted to the mean plus 1.5 times the standard deviation, and the error statistics window has been reduced to 50 macroblocks. The hardware decoding chip has been put into low-power mode, with the clock frequency reduced by 30%, and the number of parallel filters in the motion compensation unit reduced to 4. The asynchronous decoding trigger condition has been adjusted to a basic frame decoding completion rate of 70%, and the pixel fusion weights have been adjusted to 0.8 to 0.2.

[0029] The delay synchronization control module adopts a simplified version of the precision time protocol, extending the clock synchronization period to 250 milliseconds. User input event encapsulation packets are reduced to 64 bytes, and operation types are reduced to 8. Sequence number allocation frequency is reduced to 500 per millisecond. The feedforward control network state judgment threshold is adjusted to a round-trip delay of 100 milliseconds, the feedback control delay observer sampling period is extended to 100 milliseconds, and the motion compensation frame interpolation algorithm adopts the nearest neighbor interpolation method, controlling the generation time to within 5 milliseconds.

[0030] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A cloud game based streaming and low latency decoding system, characterized in that, The system comprises the following components: a dynamic feature extraction module: through a game engine application program interface, a game picture sequence with a frame rate not less than 60 frames per second is obtained in real time, motion estimation calculation is performed on the continuous frame sequence, a motion vector field is generated, and the proportion of pixels with a motion vector amplitude exceeding a preset threshold in a unit area is counted as a motion vector density, the root mean square error of each pixel point color component between adjacent frames is calculated as a color change rate, a target detection algorithm is used to identify key objects of a preset category in the picture, and a trajectory sequence is formed by tracking the continuous change of the center coordinates thereof; An adaptive coding control module: based on network state parameters and the dynamic characteristics, dynamically adjusting the coding rate and resolution through a reinforcement learning model, supporting hierarchical coding, the state space of the reinforcement learning model is defined as a three-dimensional vector [motion vector density, color change rate, key object trajectory curvature], the action space contains the code rate adjustment instruction set {+20%, +10%, 0%, -10%, -20%} and the resolution switching instruction set {4K, 1440p, 1080p}, the reward function is designed as Wherein is the end-to-end delay measured value, PSNR is the peak signal-to-noise ratio, PLR is the packet loss rate, α, β, γ are weight coefficients and satisfy α>β>γ>0; a prediction coding optimization module: using an AI prediction model to generate a current frame prediction result, combining intra / inter prediction coding, marking the key frame patch in the area with large prediction error and preferentially transmitting, the AI prediction model adopts a lightweight convolutional neural network with an encoder-decoder architecture, the encoder includes 3 convolutional layers and 2 max pooling layers, the decoder includes 2 deconvolutional layers and 1 output convolutional layer, the network input is 5 consecutive 128x128 pixel YUV420 format image blocks, and the output is the prediction result of the corresponding image block of the current frame; a hardware acceleration decoding module: accelerating decoding through a special hardware decoding chip, using asynchronous fast decoding for key frame patches, the special hardware decoding chip integrates a programmable pipeline architecture, including an entropy decoding unit, an inverse quantization unit, an inverse transformation unit and a motion compensation unit, the motion compensation unit has 8 parallel interpolation filters built-in, supporting bilinear interpolation calculation with 1 / 4 pixel precision; a delay synchronization control module: synchronizing the decoding frame and the user input event, controlling the end-to-end total delay ≤50 milliseconds, using the precision time protocol to realize the clock deviation between the cloud and the terminal ≤1 millisecond, encapsulating the user input event into a 96-byte binary protocol data unit, including an 8-byte timestamp, a 4-byte operation type identifier and an 84-byte operation parameter, and the binding relationship between the input event and the video frame is established through a 64-bit serial number.

2. The cloud gaming-based streaming and low-latency decoding system of claim 1, wherein, In the dynamic feature extraction module, the target detection algorithm uses a YOLOv5 model to identify three types of key objects, namely a player character, a skill special effect and an environmental object, tracks the continuous change of the center coordinates thereof through a Kalman filter, and forms a trajectory sequence containing position coordinates, motion speed and direction angle, and the trajectory data is updated at a frequency of 30 sampling points per frame.

3. The cloud gaming-based streaming and low-latency decoding system of claim 1, wherein, In the adaptive encoding control module, the Base layer of the hierarchical encoding uses a fixed value of a quantization parameter 32 for encoding, discrete cosine transform coefficients containing direct current components and the first three alternating current components, and motion vector basic information at a 16x16 pixel block level, and the data packet is marked as 46 in the differentiated service code point; the Enhancement layer uses a variable value of a quantization parameter 24 for encoding, supplements high-frequency detail coefficients and fine motion vectors at a 4x4 pixel block level, and the data packet is marked as 26 in the differentiated service code point.

4. The cloud gaming-based streaming and low-latency decoding system of claim 1, wherein, In the prediction coding optimization module, the full search algorithm of 35 prediction modes is used for the intra prediction, the DC mode or the planar mode is forced to be used for the region marked as the key frame patch to reduce the calculation complexity; the improved diamond search algorithm is used for the inter prediction, and the search range is limited within the 16x16 pixel region; the marking condition of the key frame patch is that the prediction error exceeds the threshold wherein μ is the global prediction error mean and σ is the standard deviation.

5. The cloud gaming-based streaming and low-latency decoding system of claim 1, wherein, In the hardware acceleration decoding module, an asynchronous fast decoding mechanism allocates an independent decoding channel for a key frame patch, starts the patch frame decoding process as soon as the basic frame decoding completion rate reaches 80%, the basic frame decoding uses a mainstream pipeline, the patch frame decoding uses a standby pipeline, and the two are fused at a pixel level in a frame buffer area, the fusion process uses an alpha blending algorithm, the basic frame pixel weight is set to 0.7, and the patch frame pixel weight is set to 0.

3.

6. The cloud gaming-based streaming and low-latency decoding system of claim 1, wherein, The delay synchronization control module, the end-to-end delay control strategy comprises a feedforward control and a feedback control double loop: the feedforward control dynamically adjusts the encoding complexity according to a network state prediction model, and the bidirectional predictive frame encoding is closed when the round-trip delay exceeds 80 milliseconds; the feedback control is based on the output of the delay observer, and the terminal enables the motion compensation frame interpolation algorithm when the end-to-end total delay exceeds 60 milliseconds, and the interpolation weight coefficient .

7. A cloud game based streaming and low latency decoding method, characterized in that, The method comprises the following steps: S1, extracting dynamic features of a game picture, including motion vector density, color change rate and key object trajectory; S2, based on network state parameters and dynamic features, dynamically adjusting encoding code rate and resolution through a reinforcement learning model, supporting hierarchical encoding; S3, generating a current frame prediction result using an AI prediction model, combining intra / inter prediction encoding, marking a region with large prediction error as a key frame patch and preferentially transmitting the key frame patch; S4, accelerating decoding through a dedicated hardware decoding chip, and using an asynchronous fast decoding for the key frame patch; S5, synchronizing the decoded frame and user input events to control the end-to-end total delay to be less than or equal to 50 milliseconds.

8. The cloud gaming based streaming and low latency decoding method of claim 7, wherein, In the step S1, a game picture sequence with a frame rate not less than 60 frames per second is obtained in real time through a game engine application program interface; motion estimation calculation is performed on the continuous frame sequence to generate a motion vector field and count the proportion of pixels with a motion vector amplitude exceeding 8 pixels per unit area as a motion vector density; the root mean square error of the luminance component and the chrominance component in the YUV color space of each pixel point between adjacent frames is calculated as a color change rate; three types of key objects, i.e., a player character, a skill special effect and an environmental object, are identified in the picture based on a YOLOv5 target detection algorithm, and a trajectory sequence is formed by tracking the continuous changes of the center coordinates of the objects through a Kalman filter.

9. The cloud gaming based streaming and low latency decoding method of claim 7, wherein, In the step S2, the state space of the reinforcement learning model is defined as a three-dimensional vector [motion vector density, color change rate, key object trajectory curvature], the action space includes the code rate adjustment instruction set {+20%, +10%, 0%, -10%, -20%} and the resolution switching instruction set {4K, 1440p, 1080p}, and the reward function is designed as where the weight coefficients are set as a=0.6, b=0.3, g=0.1, and the model performs a policy update every 200 milliseconds.

10. The cloud gaming based streaming and low latency decoding method of claim 7, wherein, In the step S4, a special hardware decoding chip is integrated with a programmable pipeline architecture, including an entropy decoding unit, an inverse quantization unit, an inverse transformation unit and a motion compensation unit; the motion compensation unit is built-in with 8 parallel interpolation filters supporting bilinear interpolation calculation with a 1 / 4 pixel precision; an asynchronous fast decoding mechanism allocates independent decoding channels for key frame patches, and patch frame decoding is started immediately when the basic frame decoding completion rate reaches 80%, and the two are fused at a pixel level in a frame buffer area.

Images