Participant-Specific Video Compression for Low-Bandwidth Videoconferencing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional video encoding schemes for live streaming fail to maintain high perception quality when buffering is unavailable and bandwidth is low, as they rely solely on spatial and temporal redundancies without utilizing participant-specific knowledge.
Innovation Solution
Implementing participant-specific video encoding and decoding techniques using machine learning to minimize redundancies beyond spatial and temporal similarities, leveraging speaker-specific characteristics such as facial expressions and gestures for enhanced compression and quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional video encoding schemes are used for live streaming, then the encoding process is simple and relies on standard spatial and temporal redundancies, but the perception quality deteriorates when buffering is unavailable and bandwidth is low
Solution Approach 1:
The system performs preliminary training of machine learning models using historical video data before actual live streaming encoding. This pre-training phase allows the encoder to learn participant-specific characteristics in advance, so that during live streaming, the pre-trained models can be directly applied to maintain high perception quality without real-time training overhead.
Solution Approach 2:
The encoding system dynamically adapts by switching between different encoding modes based on available bandwidth and buffering conditions. When bandwidth is low and buffering unavailable, the system activates participant-specific ML-based encoding. The system also dynamically updates models with new video data during streaming to adapt to changing participant appearances and behaviors.
2Productivity
If machine learning-based participant-specific encoding is implemented, then compression ratio and perception quality improve in live streaming without buffering, but the device complexity and processing requirements increase
Solution Approach 1:
The video encoding process is segmented into distinct phases: a training phase using historical data to build participant-specific models, and an inference phase during live streaming that applies these models. This segmentation allows computationally intensive model training to occur offline, while online encoding uses lighter-weight inference operations.
Solution Approach 2:
Participant-specific encoding models are trained in advance using historical video data before live streaming begins. This preliminary training captures individual characteristics such as facial features, gestures, and expressions, enabling the system to achieve high compression ratios during streaming without performing heavy training computations in real-time.
3Loss of information
If conventional encoding relying on spatial and temporal redundancies is used, then the encoding process is fast and computationally efficient, but the compression effectiveness is limited without participant-specific knowledge
Solution Approach 1:
The system continuously receives feedback from the streaming process, including decoded video quality metrics and bandwidth conditions. This feedback is used to adjust encoding parameters and update participant-specific models in real-time, allowing the system to optimize compression effectiveness while adapting to changing conditions during live streaming.
Solution Approach 2:
The system performs preliminary analysis of participant-specific characteristics such as facial expressions, gestures, and typical movements during a training phase. This advance preparation creates detailed participant profiles that enable more effective compression during live streaming by anticipating and efficiently encoding expected participant behaviors and appearances.
Data Source
AI summary
A computer-implemented method includes receiving, from a first device of a first participant of a videoconference, a first compressed video stream and a customized first machine-trained decoding model. The first compressed video stream is compressed using a customized first machine-trained encoding model. The first decoding model and the first compressed video stream are transmitted to a second device of a second participant in the videoconference. A second compressed video stream and a customized second machine-trained decoding model are received from the second device. The second compressed video stream is compressed using a customized second machine-trained encoding model. The second decoding model and second compressed video stream are transmitted to the first device by the server. The first compressed video stream is decoded on the second device using the first decoding model and the second compressed video stream is decoded on the first device using the second decoding model.