Low Frame Rate Video Conferencing via Face Morphing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video conferencing systems are inefficient at very low bit rates, as they degrade image quality due to bandwidth constraints, especially in multi-party conferences, and struggle to maintain natural-looking results at low bit rates.
Innovation Solution
The system employs image cropping and morphing to reduce frame rates, focusing on transmitting only the face region, using face tracking to align frames and applying image morphing to align eye and mouth positions, thereby reducing data transmission while maintaining visual quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If conventional video compression techniques are used to reduce bandwidth, then bandwidth requirements are reduced, but image quality degrades significantly
Solution Approach 1:
The patent extracts and transmits only the face region from the full video frame, separating the important information (face) from the redundant background. This extraction approach allows significant bandwidth reduction while maintaining face image quality, as the background is discarded and only the essential face portion is compressed and transmitted.
Solution Approach 2:
The video stream is segmented into face region and background region, with only the face region being transmitted at high quality. The face region is further segmented into key features (eyes, mouth, nose) that are tracked and morphed to maintain natural appearance at low frame rates.
2Loss of energy
If frame rate is reduced to lower bit rate, then bandwidth requirements are reduced, but visual quality and natural appearance deteriorate
Solution Approach 1:
The system performs preliminary face tracking and feature point identification on each frame before compression. By pre-locating the eyes, mouth, and nose positions, the system can apply targeted morphing operations that preserve the natural appearance of facial features even when frames are skipped, maintaining visual quality at reduced bit rates.
Solution Approach 2:
The patent changes the parameter of frame rate from standard values to very low values (e.g., 1-2 fps), but compensates by applying image morphing techniques that adjust facial feature parameters (eye position, mouth position, nose position) to maintain natural appearance. This parameter change approach allows extreme bandwidth reduction while preserving visual quality through intelligent morphing.
3Loss of information
If face tracking and morphing are applied to align frames, then interframe difference is reduced, but processing complexity increases
Solution Approach 1:
Instead of applying complex processing to the entire frame, the patent applies face tracking and morphing operations only to the face region, and specifically only to key feature points (eyes, mouth, nose). This local quality approach reduces processing complexity compared to full-frame processing, while still achieving significant interframe difference reduction in the transmitted face region.
4Loss of energy
If only face region is transmitted instead of full frame, then bandwidth usage is reduced, but system complexity increases due to face detection and tracking
Solution Approach 1:
The patent extracts the face region from the full video frame using face detection algorithms, then transmits only this extracted region. This extraction process, while adding some processing complexity, achieves substantial bandwidth reduction by eliminating the background. The extracted face region is then processed with tracking and morphing to maintain quality.
Data Source
AI summary
A real-time low frame-rate video compression system and method that allows the user to perform face-to-face communication through an extremely low bandwidth network. The system and method employs image cropping and morphing to reduce frame rates. At the encoder side, the system is able to automatically select only a few good faces from the original sequence with high visual quality and compress and transmit them. At the decoder side, the system use image-morphing based rendering method to generate a normal frame-rate video. Experimental results show that the system is superior to more traditional video codecs for low bit-rate face-to-face communication.