Face Video Decoding With Geometric Data and Layered Face Units
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding technologies face challenges in efficiently processing large amounts of digital video data while maintaining image quality and reducing processing delays and resource consumption.
Innovation Solution
A decoder architecture that separates the encoding and decoding of face images into base and enhancement data units, utilizing a generative model to generate face videos from these units, thereby reducing code amount and delay.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional video coding technologies are used to process digital video data, then image quality can be maintained, but processing delays and resource consumption increase
Solution Approach 1:
The video data is segmented into base data units and enhancement data units. Base data units contain essential information for basic video reconstruction, while enhancement data units contain additional details for improved quality. This segmentation allows the decoder to process base data units first to maintain acceptable image quality while reducing processing delays, and optionally process enhancement data units later to improve quality when resources are available.
Solution Approach 2:
The patent applies different quality levels to different parts of the video data. Base data units provide a baseline quality level that ensures acceptable image reconstruction under all conditions. Enhancement data units provide localized quality improvements in specific regions or for specific frames. This allows the system to maintain reliable image quality while optimizing processing timing by prioritizing base data units.
2Reliability
If conventional video coding technologies are used to process digital video data, then image quality can be maintained, but resource consumption increases
Solution Approach 1:
The video coding system segments data into base and enhancement components that can be processed independently. The base data units require minimal processing resources to decode and provide acceptable image quality. Enhancement data units consume additional resources but can be processed selectively based on available resources. This segmentation enables the system to maintain image quality reliability while adapting resource consumption to available capacity.
Solution Approach 2:
The patent changes the parameter of data representation by using base data units with essential information and enhancement data units with supplementary information. This parameter change allows the decoder to adjust processing resource usage by selectively decoding enhancement data units based on available resources, while base data units ensure minimum quality requirements are always met.
3Productivity
If base data units and enhancement data units are separately decoded and processed, then coding efficiency is improved, but device complexity increases
Solution Approach 1:
The decoder is segmented into functional modules that handle base data units and enhancement data units separately. This modular segmentation improves coding efficiency by allowing independent processing of different data types. The increased device complexity is managed through systematic organization of these modules, where each module has a specific function and interfaces are well-defined, making the complexity manageable and maintainable.
Solution Approach 2:
The patent introduces intermediary data structures and processing stages that bridge the base data unit decoding and enhancement data unit processing. These intermediaries manage the interaction between different decoding modules, coordinating data flow and synchronization. While this increases device complexity, it enables improved coding efficiency through optimized processing pipelines and parallel operation of different data unit types.
Data Source
AI summary
A decoder includes memory and circuitry coupled to the memory. Using the memory, the circuitry: decodes, from a bitstream, a base data unit of a face image related to a face video and one or more enhancement data units of the face image; decodes, from the bitstream, geometric information corresponding to each of frames of the face video; and generates the face video from the base data unit, the one or more enhancement data units, and the geometric information. In the bitstream, the base data unit is added to a data set corresponding to a first frame that is a frame of the face video. In the bitstream, the one or more enhancement data units are added to one or more data sets corresponding to one or more second frames of the face video.