Stereo Video Coding With Parallel Autoencoders for Low Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding techniques struggle to efficiently compress stereo video data, particularly in low-latency scenarios, as they are not optimized for same-view temporal redundancy and cross-view spatial redundancy, leading to increased demands on communication networks and devices.
Innovation Solution
A machine learning-based video codec that utilizes parallel autoencoders to compress left and right views of a scene in parallel, leveraging inter-view information to minimize latency and optimize for rate-distortion performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional sequential video coding techniques are used to compress stereo video, then compression can be achieved, but latency increases and compression efficiency deteriorates due to failure to exploit temporal and cross-view redundancies
Solution Approach 1:
The patent segments the stereo video compression task into two independent parallel autoencoders: one for the left view and one for the right view. Each autoencoder processes its respective view independently through separate encoder and decoder networks, allowing simultaneous compression without sequential dependencies. This segmentation enables both views to be compressed in parallel, reducing latency while maintaining compression efficiency by exploiting temporal redundancies within each view independently.
Solution Approach 2:
The patent merges the compression of both stereo views into a unified parallel processing framework where left and right view autoencoders operate simultaneously. By combining the processing of multiple views in parallel rather than sequentially, the system achieves lower latency while maintaining high compression efficiency. The merged framework processes temporal redundancies across both views concurrently, improving overall productivity without increasing latency.
2Manufacturing precision
If video quality requirements (high fidelity, resolution, frame rate) are increased to meet consumer demands, then video quality improves, but the amount of video data increases placing greater burden on communication networks and devices
Solution Approach 1:
The patent transforms the video data from spatial domain to latent space representation through neural network encoders. This parameter transformation compresses high-resolution, high-frame-rate video data into compact latent representations that capture essential visual information with significantly reduced data volume. The encoder networks learn optimal parameter transformations that maintain video quality metrics (fidelity, resolution, frame rate) while dramatically reducing the quantity of data that needs to be transmitted and processed.
Solution Approach 2:
The patent extracts only the essential visual information from high-quality video data through the autoencoder compression process. The encoder networks identify and extract key features and temporal patterns while discarding redundant information, producing compact latent representations. This extraction process maintains the essential quality attributes (fidelity, resolution, frame rate) while removing unnecessary data volume, reducing the burden on communication networks and processing devices.
3Productivity
If conventional video coding methods are used that do not optimize for inter-view information, then implementation is simpler, but compression efficiency deteriorates due to increased video data volume
Solution Approach 1:
The patent implements self-service compression where each view's autoencoder independently learns and exploits temporal redundancies within its own view data. The left view encoder processes left view temporal patterns, and the right view encoder processes right view temporal patterns autonomously without requiring complex inter-view coordination. This self-service approach achieves high compression efficiency by capturing temporal redundancies while keeping the system relatively simple compared to methods requiring complex inter-view optimization.
Data Source
AI summary
Systems and techniques are described herein for processing video data. For example, a machine-learning based stereo video coding system can obtain video data including at least a right-view image of a right view of a scene and a left-view image of a left view of the scene. The machine-learning based stereo video coding system can compress the right-view image and the left-view image in parallel to generate a latent representation of the right-view image and the left-view image. The right-view image and the left-view image can be compressed in parallel based on inter-view information between the right-view image and the left-view image, determined using one or more parallel autoencoders.