Video Compression Using Conditional GAN and Soft Edge Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video compression techniques using deep neural networks require a large number of frames for training and a reference frame for interpolation, leading to deficient compression ratios and inefficient video reconstruction.
Innovation Solution
A two-stage video compression framework using a conditional Generative Adversarial Network (GAN) that partitions video frames into key and non-key frames, where only key frames are used for training, and soft edge information is used to reconstruct non-key frames without the need for interpolation, allowing for better compression ratios and enhanced video quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If deep neural networks are used for video compression with motion estimation as interpolation, then video reconstruction quality is improved, but compression ratio deteriorates due to requiring large volume of training frames and reference frames
Solution Approach 1:
The patent segments the video compression task into two distinct stages: a first stage encoder that processes key frames to generate a first representation, and a second stage encoder that processes non-key frames to generate soft edge information. This segmentation allows the system to use only key frames for training the generative model while efficiently compressing non-key frames through soft edge encoding, thereby improving compression ratio while maintaining reconstruction quality.
Solution Approach 2:
The patent extracts and processes only the essential soft edge information from non-key frames in the second stage, rather than encoding complete frame data. By extracting only the critical edge information and using a generative model to reconstruct full frames, the system achieves better compression ratios while preserving visual quality.
2Measurement precision
If reference frames are used for interpolation to reconstruct target frames, then reconstruction accuracy is improved, but system complexity increases due to frame selection and interpolation processes
Solution Approach 1:
The patent uses a generative model trained on key frames to synthesize and generate reconstructed frames from soft edge information, rather than using traditional interpolation methods that require reference frames. This copying approach allows the system to reconstruct frames without complex reference frame selection and interpolation processes, reducing system complexity while maintaining reconstruction accuracy.
3Reliability
If large volume of training video frames are used, then neural network performance is improved, but training data requirements increase leading to inefficient compression
Solution Approach 1:
The patent segments video frames into key frames and non-key frames, using only key frames for training the generative model. This segmentation strategy reduces training data requirements significantly while maintaining neural network performance, as key frames contain sufficient information for the model to learn effective compression and reconstruction patterns.
Solution Approach 2:
The patent applies partial action by using only a subset of frames (key frames) for training instead of the full video sequence. This partial training approach is sufficient to achieve good neural network performance while dramatically reducing training data requirements and improving compression efficiency.
Data Source
AI summary
A computer-implemented method, system and computer program product for compressing video. A set of video frames is partitioned into two subsets of different types of frames, a first type and a second type. The first type of frames of videos is compressed to generate a first representation by a first stage encoder. The first representation is then decoded to reconstruct the first type of frames using a first stage decoder. The second type of frames of video is compressed to generate a second representation that only contains soft edge information by a second stage encoder. A generative model corresponding to a second stage decoder is then trained using the first representation and the reconstructed first type of frames by using a discriminator employed by a machine learning system. After training the generative model, it generates reconstructed first and second types of frames using the soft edge information.