Neural Network Pre-Filter for Streaming Entropy Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional video game streaming systems face issues with dynamic resolution changes, leading to high bit rate I-frames during strained network conditions, resulting in incomplete or delayed frames, reduced picture quality, and increased latency, negatively impacting user experience.
Innovation Solution
A neural network-based pre-filter dynamically adapts scene entropy in response to changing network conditions, reducing bit rates by filtering frames before streaming, without requiring resolution changes or decoder re-initialization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If dynamic resolution changes are implemented to address network conditions, then adaptability to network capacity is improved, but bit rate increases due to high-complexity I-frames
Solution Approach 1:
The patent applies preliminary action by pre-filtering video frames using a neural network before encoding and transmission. The pre-filter reduces scene complexity and entropy in advance, so that when I-frames are needed for resolution changes, their bit rate requirement is already reduced. This prepares the content beforehand to avoid the contradiction between adaptability and bit rate consumption.
Solution Approach 2:
The patent changes the parameter of frame complexity/entropy through neural network pre-filtering. By adjusting the filtering strength based on network conditions, the system can reduce the entropy parameter of frames, thereby lowering the bit rate required for transmission while maintaining adaptability to different network capacities.
2Adaptability or versatility
If I-frames are transmitted during strained network conditions, then resolution transition capability is improved, but frame completion rate deteriorates due to packet loss
Solution Approach 1:
The neural network pre-filter is applied in advance to reduce the size and complexity of I-frames before transmission. This preliminary reduction makes the I-frames more resilient to packet loss during strained network conditions, improving the likelihood of complete frame reception while maintaining resolution transition capability.
Solution Approach 2:
The patent converts the harmful effect of strained network conditions into a benefit by using the neural network to identify and preserve only the most important visual information in I-frames. This selective preservation ensures that even under packet loss conditions, the essential scene information is maintained, turning the network strain into an opportunity for optimized transmission.
3Manufacturing precision
If I-frames with high bit rate are transmitted, then spatial compression is improved, but transmission time increases due to channel capacity constraints
Solution Approach 1:
The patent changes the parameter of frame entropy through neural network pre-filtering, reducing the complexity of spatial details while preserving important visual information. This parameter change allows for efficient compression that maintains acceptable spatial quality while significantly reducing the bit rate and subsequent transmission time over constrained channels.
Solution Approach 2:
The neural network applies local quality optimization by selectively preserving important visual features (such as edges, textures, and significant objects) while compressing less important areas. This localized preservation maintains spatial compression quality where it matters most while reducing overall bit rate and transmission time.
4Quantity of substance
If frame resolution is reduced to lower bit rate, then network bandwidth consumption is improved, but picture quality deteriorates
Solution Approach 1:
Instead of changing the resolution parameter, the patent changes the entropy/complexity parameter of the frame content through neural network pre-filtering. This allows the same resolution to be transmitted at a lower bit rate by reducing unnecessary detail, thereby consuming less network bandwidth while maintaining acceptable picture quality.
Solution Approach 2:
The patent segments the visual information into important and less important components through the neural network pre-filter. By separating and selectively transmitting only the essential visual information at reduced complexity, the system achieves lower bandwidth consumption while preserving the quality of critical picture elements.
Data Source
AI summary
In various examples, a deep neural network (DNN) based pre-filter for content streaming applications is used to dynamically adapt scene entropy (e.g., complexity) in response to changing network or system conditions of an end-user device. For example, where network and/or system performance issues or degradation are identified, the DNN may be implemented as a frame pre-filter to reduce the complexity or entropy of the frame prior to streaming-thereby allowing the frame to be streamed at a reduced bit rate without requiring a change in resolution. The DNN-based pre-filter may be tuned to maintain image detail along object, boundary, and/or surface edges such that scene navigation—such as by a user participating in an instance of an application—may be easier and more natural to the user.