Videoconferencing Facial Image Rectification via Polygon Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing videoconferencing technologies face challenges in maintaining image quality due to high network error rates or low data transmission rates, particularly for images containing faces, which current compensation methods have not adequately addressed.
Innovation Solution
A method involving the detection of feature landmarks in image frames, partitioning regions into polygons, translating image data between corresponding polygons, and forming composite frames using neural processing units to enhance image quality, specifically utilizing U-Net and VDSR architectures for image rectification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If conventional image transmission methods are used over networks with high error rates or low data transmission rates, then network compatibility is maintained, but image quality deteriorates
Solution Approach 1:
The patent segments the facial region into multiple polygons based on detected feature landmarks. This segmentation allows selective processing and replacement of only the facial region using reference images, rather than processing the entire image. The polygon-based segmentation enables precise mapping and replacement of facial features while maintaining other image regions unchanged, thereby improving image quality under poor network conditions without requiring complete image retransmission.
Solution Approach 2:
The patent performs preliminary action by detecting feature landmarks and partitioning the facial region into polygons before image transmission or replacement. This pre-processing establishes a structural framework that guides subsequent image data replacement. By pre-identifying the facial region and its polygonal subdivisions, the system can efficiently replace only necessary facial image data using reference images, improving quality without transmitting complete high-resolution images.
2Manufacturing precision
If image data is transmitted with higher resolution to maintain quality, then image quality improves, but data transmission requirements increase
Solution Approach 1:
The patent applies local quality by focusing enhancement efforts specifically on the facial region rather than the entire image. By detecting feature landmarks and partitioning only the facial area into polygons, the system replaces image data locally within these polygons using reference images. This localized approach improves facial image quality while avoiding the need to transmit or process entire high-resolution images, thereby reducing overall data transmission requirements.
Solution Approach 2:
The patent segments the image into multiple regions, with special processing applied only to the facial region partitioned into polygons. This segmentation enables selective replacement of facial image data using reference images, while other image regions remain unchanged. The segmentation strategy allows the system to improve perceived image quality (particularly facial features) without increasing the transmission volume of the complete image.
3Measurement precision
If feature landmark detection and polygon partitioning are implemented, then image rectification accuracy improves, but processing complexity increases
Solution Approach 1:
The patent segments the facial region into multiple polygons based on detected feature landmarks. This segmentation, while adding processing steps, enables precise mapping and replacement of facial features by establishing clear geometric boundaries. The polygon-based approach provides a structured method for identifying and replacing specific facial regions, improving rectification accuracy through systematic division of the facial area into manageable geometric units.
Solution Approach 2:
The patent introduces feature landmarks as intermediary points that mediate between the raw image data and the final rectified output. These landmarks serve as reference points for detecting facial features and defining polygon boundaries. By using landmarks as intermediaries, the system achieves accurate facial feature detection and mapping without requiring complex direct image transformation algorithms, thereby managing processing complexity through the use of intermediate reference structures.
Data Source
AI summary
A real-time method (600) for enhancing facial images (102). Degraded images (102) of a person—such as might be transmitted during a videoconference—are rectified based on a single high definition reference image (604) of a person who is talking. Facial landmarks (501) are used to map (210) image data from the reference image (604) to an intervening image (622) having a landmark configuration like that in a degraded image (102). The degraded images (102) and their corresponding intervening images (622) are blended using an artificial neural network (800, 900) to produce high-quality images (108) of the person who is speaking during a videoconference.