Interactive Frame Prediction Model for Streaming Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current game streaming services face challenges in providing high-resolution frames in real-time due to the limitations of client devices in processing high-specification games, necessitating innovative methods to enhance streaming data transmission efficiency.
Innovation Solution
A streaming system that employs an interactive frame prediction model trained on user input and metadata, allowing for adaptive encoding and decoding of streaming data to improve compression rates and facilitate real-time transmission of high-resolution frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high-resolution frames are transmitted in real-time, then image quality is improved, but transmission bandwidth and processing requirements increase
Solution Approach 1:
The system performs frame prediction in advance by training a neural network model on streaming data, user inputs, and metadata to generate predicted frames before actual transmission is needed. This preliminary action reduces the amount of data that needs to be transmitted in real-time, as only the differences between predicted and actual frames need to be sent, thereby improving transmission efficiency while maintaining high image quality
Solution Approach 2:
The patent introduces an interactive frame prediction model as an intermediary between the game streaming server and the client device. This model processes streaming data, user inputs, and metadata to generate predicted frames that serve as a reference for compression. The intermediary reduces the bandwidth requirement by transmitting only residual data rather than complete high-resolution frames, thus resolving the contradiction between image quality and transmission efficiency
2Productivity
If compression rate is increased, then transmission efficiency is improved, but image quality may deteriorate
Solution Approach 1:
The system employs feedback mechanisms where the frame prediction model is continuously trained and refined based on user inputs, metadata, and actual streaming data. The model adjusts its predictions based on feedback from user interactions and system performance, allowing it to maintain high prediction accuracy. This feedback loop ensures that compression can be aggressive while still preserving image quality, as the model adapts to minimize prediction errors
Solution Approach 2:
The patent utilizes parameter changes by dynamically adjusting the prediction model based on varying conditions such as user input patterns, game state metadata, and streaming data characteristics. By changing the parameters and weights of the neural network model in response to different scenarios, the system optimizes the balance between compression rate and image quality, achieving high transmission efficiency without sacrificing visual fidelity
3Productivity
If client device processes high-specification games, then game performance is improved, but device complexity and power consumption increase
Solution Approach 1:
The patent extracts the complex processing workload from the client device by implementing frame prediction and encoding on the server side. The server trains the interactive frame prediction model and generates predicted frames, then transmits only compressed residual data to the client. This extraction of processing requirements allows the client device to have lower specifications while still achieving high game performance, as the heavy computational tasks are performed remotely
Data Source
AI summary
A streaming system includes a streaming server and a client device. The streaming server is configured to train an interactive frame prediction model based on streaming data, a user input and metadata associated with the user input, encode the streaming data by selectively using a predicted frame generated based on the trained interactive frame prediction model and transmit the trained interactive frame prediction model and the encoded streaming data. The client device is configured to receive the trained interactive frame prediction model and the encoded streaming data, and decode the encoded streaming data based on the trained interactive frame prediction model to provide recovered streaming data to a user.