Face Video Decoder Using Base and Enhancement Frame Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding technologies face challenges in improving coding efficiency, enhancing image quality, reducing processing amounts, and circuit scales, and appropriately selecting elements or operations such as filters, blocks, motion vectors, and reference pictures.
Innovation Solution
A decoder configuration that includes memory and circuitry to decode base and enhancement data units of face images, along with geometric information, using a generative model to generate face videos, allowing separate decoding of these units across frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional video coding schemes are used, then processing amounts and circuit scales are reduced, but coding efficiency and image quality cannot be sufficiently improved
Solution Approach 1:
The patent segments the face image data into base data units and enhancement data units, which are separately decoded and processed. This segmentation allows the system to handle different types of data with appropriate processing methods, improving coding efficiency without proportionally increasing processing complexity.
Solution Approach 2:
The patent introduces geometric information as an intermediary element that describes spatial relationships and attributes of face regions. This intermediary data structure enables more efficient encoding and decoding by providing structured information about face geometry, reducing the processing burden while improving coding efficiency.
2Manufacturing precision
If conventional video coding schemes are used, then processing amounts and circuit scales are reduced, but image quality cannot be sufficiently enhanced
Solution Approach 1:
The patent applies local quality enhancement by focusing processing resources on face regions specifically. The base data units and enhancement data units are targeted at face image portions, allowing high-quality reconstruction of faces while using less processing power overall compared to processing entire video frames uniformly.
Solution Approach 2:
The patent performs preliminary encoding of geometric information and face data units in advance, organizing them into structured formats before decoding. This preliminary organization reduces the complexity of the decoding process and enables high-quality image reconstruction with reduced processing requirements.
3Productivity
If base data units and enhancement data units are separately decoded across frames, then coding efficiency is improved, but processing delays may increase
Solution Approach 1:
The patent employs periodic action by alternately processing base data units and enhancement data units in a structured sequence. This periodic processing pattern allows the decoder to efficiently manage different data types while maintaining a predictable processing rhythm that minimizes delays.
Solution Approach 2:
The patent performs preliminary decoding of base data units before enhancement data units, establishing a temporal order that optimizes processing efficiency. By preparing and decoding foundational data first, the system reduces overall processing delays while maintaining high coding efficiency.
Data Source
Figure 1
Figure 2(a)~2(f)
Figure 3
AI summary
A decoder (200) includes memory (252) and circuitry (251) coupled to the memory (252). Using the memory (252), the circuitry (251): decodes, from a bitstream, a base data unit of a face image related to a face video and one or more enhancement data units of the face image; decodes, from the bitstream, geometric information corresponding to each of frames of the face video; and generates the face video from the base data unit, the one or more enhancement data units, and the geometric information. In the bitstream, the base data unit is added to a data set corresponding to a first frame that is a frame of the face video. In the bitstream, the one or more enhancement data units are added to one or more data sets corresponding to one or more second frames of the face video.