Video Compression Spatial Temporal Resolution Tradeoffs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video systems are limited in their ability to faithfully represent scenes with high motion due to low frame rates, resulting in artifacts like motion blur, judder, and strobing, and are constrained by bottlenecks in processing power and data rate, making it costly to upgrade to handle higher frame rates.
Innovation Solution
The method involves temporally partitioning input video into pulses and determining a compressed representation for each pulse by trading off temporal and spatial resolution, optimizing perceived video quality while reducing data rate, and reconstructing video at high frame rates by upsampling data close to the display device.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the frame rate is increased to avoid motion artifacts, then motion perception quality is improved, but data rate and processing power requirements increase
Solution Approach 1:
The video stream is segmented into multiple channels based on motion characteristics. High-motion regions are separated from low-motion regions, allowing different frame rates to be applied to different segments. This enables the system to provide high frame rates only where needed for motion perception, while reducing overall data rate by using lower frame rates for static or slowly moving regions.
Solution Approach 2:
Different quality parameters (frame rates) are applied to different regions of the video based on local motion characteristics. High-motion regions receive higher frame rates to maintain motion perception quality, while low-motion regions receive lower frame rates to reduce data rate. This local adaptation resolves the contradiction by optimizing quality only where necessary.
2Reliability
If the frame rate is increased to avoid motion artifacts, then motion perception quality is improved, but processing power requirements increase
Solution Approach 1:
The video processing workload is segmented into multiple channels with different frame rates. By processing only the necessary number of frames for each region based on motion characteristics, the system reduces overall processing power requirements while maintaining motion perception quality in high-motion regions.
Solution Approach 2:
The system applies high frame rates partially, only to regions where motion perception is critical, rather than processing the entire video stream at high frame rates. This partial action approach maintains motion perception quality where needed while significantly reducing total processing power consumption.
3Measurement precision
If spatial resolution is maintained at high frame rates, then image detail is improved, but data rate increases
Solution Approach 1:
The system applies different spatial resolutions to different regions based on motion characteristics. High-motion regions are processed with lower spatial resolution at high frame rates, while low-motion regions maintain high spatial resolution at lower frame rates. This local quality adaptation resolves the contradiction by maintaining image detail where motion is minimal while reducing data rate through resolution reduction in high-motion areas.
Solution Approach 2:
The spatial resolution parameter is made dynamic, changing based on local motion characteristics rather than being fixed throughout the video stream. This dynamic adjustment allows the system to optimize the balance between image detail and data rate by adapting resolution to local content requirements.
Data Source
AI summary
Compression transforming video into a compressed representation (which typically can be delivered at a capped pixel rate compatible with conventional video systems), including by generating spatially blended pixels and temporally blended pixels (e.g., temporally and spatially blended pixels) of the video, and determining a subset of the blended pixels for inclusion in the compressed representation including by assessing quality of reconstructed video determined from candidate sets of the blended pixels. Trade-offs may be made between temporal resolution and spatial resolution of regions of reconstructed video determined by the compressed representation to optimize perceived video quality while reducing the data rate. The compressed data may be packed into frames. A reconstruction method generates video from a compressed representation using metadata indicative of at least one reconstruction parameter for spatial regions of the reconstructed video.