Neural Network Video Pre-Post Processing for Bandwidth Quality Trade-off
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video compression techniques are inadequate to efficiently deliver high-quality video content due to limitations in network bandwidth and the need for reduced resolution to avoid buffering delays, resulting in suboptimal video quality on high-resolution displays, especially in live broadcasts where processing resources are limited.
Innovation Solution
A method involving jointly training pre-processing and post-processing neural networks to optimize visual data encoding and decoding, using a differential approximation of a standard codec process to minimize reconstruction error and bit usage, allowing for efficient transmission and reconstruction of high-quality video data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If video is transmitted at high resolution to match display capabilities, then video quality is improved, but network bandwidth requirements increase and buffering delays occur
Solution Approach 1:
The patent applies preliminary action by performing super-resolution processing on the transmitted video signal before delivery to the display device. Instead of transmitting high-resolution video directly (which would require excessive bandwidth), the system transmits lower-resolution video that is then enhanced through neural network-based super-resolution processing at the receiving end, thereby achieving high visual quality without the corresponding data transmission burden
Solution Approach 2:
The patent introduces an intermediary processing stage using neural networks and super-resolution algorithms between the transmitted video signal and the final display output. This intermediary system acts as a mediator that transforms low-resolution input into high-resolution output, allowing the system to decouple the transmission resolution from the display resolution, thus resolving the contradiction between maintaining video quality and reducing data transmission volume
2Productivity
If video resolution is reduced to match available bandwidth, then data transmission efficiency is improved, but video quality deteriorates on high-resolution displays
Solution Approach 1:
The patent applies parameter changes by dynamically adjusting the resolution parameters of the transmitted video signal based on available bandwidth conditions, while simultaneously employing super-resolution processing to restore high visual quality. The system changes the transmission parameter (resolution) to optimize data efficiency, then compensates through algorithmic parameter transformation (super-resolution) to maintain or enhance perceived video quality on high-resolution displays
Solution Approach 2:
The patent employs a composite approach by combining traditional video compression techniques with neural network-based super-resolution processing. This composite system integrates multiple processing stages: initial compression for efficient transmission, followed by neural network enhancement to reconstruct high-resolution details. The combination allows the system to achieve both data transmission efficiency and high video quality, resolving the contradiction between these two opposing requirements
3Quantity of substance
If standard compression techniques are used to reduce data size, then transmission bandwidth is optimized, but reconstruction quality and visual fidelity are lost
Solution Approach 1:
The patent replaces traditional mechanical compression algorithms with neural network-based intelligent processing systems. Instead of using conventional compression techniques that permanently discard visual information to reduce data size, the system employs trained neural networks that learn to preserve essential visual information while achieving efficient compression. The neural networks can reconstruct high-fidelity visual data from compressed representations, thereby reducing data size without suffering the same information loss as traditional methods
Solution Approach 2:
The patent introduces neural network processing as an intermediary between compression and decompression stages. This intermediary system uses learned models to intelligently preserve and reconstruct visual information that would otherwise be lost in standard compression. The neural network acts as a mediator that maintains visual fidelity by predicting and restoring missing details, thereby reducing information loss while achieving efficient data size reduction
Data Source
Figure 1a~1b
Figure 2
AI summary
The present invention relates to a method for training a plurality of visual processing algorithms for processing visual data. Thee method comprising the steps of using a pre-processing hierarchical algorithm to process the visual data prior to encoding the visual data in visual data processing, and using a post-processing hierarchical algorithm to further process the visual data following decoding visual data in visual data processing. The steps of steps of encoding and decoding are performed with respect to a predetermined visual data codec and in some embodiments may be content specific.