Generative Face Video Compression: Relative Metrics, Stable GAN Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing facial video compression techniques, particularly in the Generative Face Video Compression (GFVC) standard, face challenges in achieving efficient compression without significant quality loss, especially due to unstable Generative Adversarial Network (GAN) training and the need for precise MSE thresholds in determining frame re-coding versus synthesis.
Innovation Solution
A relative difference metric is computed to determine whether a frame should be re-coded or synthesized by a generative neural network, and a two-stage training method is employed to stabilize GAN training, ensuring stable and efficient frame processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If Generative Adversarial Networks are used for frame synthesis in GFVC, then compression efficiency is improved, but training stability deteriorates
Solution Approach 1:
The training process is segmented into two distinct stages: Stage 1 trains only the generator network to produce stable feature representations, while Stage 2 introduces the discriminator network for quality enhancement. This segmentation resolves the training instability by preventing the discriminator from interfering with the generator's learning process during early training phases.
Solution Approach 2:
The generator network is trained in advance (Stage 1) before the discriminator is introduced. This preliminary training ensures the generator achieves stable feature extraction capabilities before quality enhancement begins, preventing the training instability that occurs when both networks are trained simultaneously.
2Ease of operation
If MSE threshold is used to determine frame re-coding versus synthesis, then decision simplicity is improved, but measurement precision deteriorates
Solution Approach 1:
The patent transforms the absolute MSE threshold approach into a relative difference metric that compares frame differences dynamically. Instead of using a fixed threshold, the system calculates relative changes between consecutive frames, adapting to varying video content characteristics and achieving both simplicity and precision in frame selection decisions.
3Productivity
If lossy compression is used to reduce image size, then compression ratio is improved, but image quality deteriorates
Solution Approach 1:
The patent introduces a generative neural network as an intermediary between the compressed reference frame and the target frame. Instead of directly compressing and transmitting all frames, the system uses the generative model to synthesize intermediate frames, achieving high compression ratios while maintaining image quality through intelligent generation rather than direct transmission.
Solution Approach 2:
The system changes the compression approach from traditional lossy compression to generative compression. By training the neural network to generate realistic frame content, the system achieves superior quality-retention characteristics compared to conventional lossy methods, effectively changing the compression paradigm rather than merely adjusting compression parameters.
Data Source
AI summary
Generative Face Video Compression (“GFVC”) techniques are provided to improve performance of facial video compression. A computing system is configured to compute a relative difference metric describing differences in features between frames, and determining, based on the relative difference metric, whether a current frame can be synthesized without entropy coding, or should be re-coded. A computing system is configured to perform two-stage training to stabilize Generative Adversarial Networks (“GAN”) training in GFVC.