Multi-Source Video Stream Production With Latency Synchronization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing digital video conference systems face challenges in producing a well-formed user experience due to varying latencies, frame rates, aspect ratios, resolutions, and encodings of incoming video streams, leading to synchronization issues and increased computational burden, especially in complex multi-participant scenarios with diverse hardware and connectivity.
Innovation Solution
A method and system for producing a digital video stream that involves collecting multiple primary streams, time-synchronizing and time-delaying them to create a second produced stream, which is then published to consuming clients with controlled latency, using a central server with format-specific collecting functions and AI-driven event and pattern detection to manage dynamic conference layouts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple digital video streams with different latencies, frame rates, resolutions, and encodings are collected and processed in real-time, then the system can support diverse participant hardware and connectivity, but synchronization issues and computational burden increase
Solution Approach 1:
The system segments the complex video processing task into distinct functional modules: a collecting function that receives streams from multiple sources, a synchronizing function that aligns timing across streams, and a production function that generates output streams. This modular segmentation allows each component to handle specific aspects of the problem independently, reducing overall system complexity while maintaining support for diverse input formats
Solution Approach 2:
The patent introduces an intermediary synchronizing function that acts as a mediator between the collecting function and the production function. This intermediary component receives unsynchronized video streams, applies time-delay adjustments to align frame rates and timing, and outputs synchronized streams to the production function. The intermediary layer isolates the complexity of synchronization from both the collection and production stages, enabling the system to handle diverse hardware configurations without overwhelming computational burden
2Adaptability or versatility
If video streams are decoded and re-encoded to统一 formats, then format compatibility is improved, but latency and computational cost increase
Solution Approach 1:
The system applies local quality transformation by performing format conversion only on specific portions of the video stream that require it, rather than re-encoding entire streams. The production function selectively applies format conversions based on the output requirements of specific participant clients, maintaining original formats for streams that are already compatible. This localized approach reduces overall processing latency and computational cost while ensuring format compatibility where needed
3Stability of the object's composition
If buffer requirements are increased to handle variable frame rates and resolutions, then memory allocation stability is improved, but latency increases
Solution Approach 1:
The system implements dynamic buffer management where buffer sizes and allocation are adjusted in real-time based on the actual characteristics of incoming video streams. The synchronizing function continuously monitors frame rates and resolutions of input streams and dynamically adjusts time-delay parameters and buffer requirements accordingly. This dynamic adaptation allows the system to maintain memory allocation stability for streams with variable parameters without consistently over-provisioning buffers, thereby minimizing latency while ensuring stable processing
Data Source
AI summary
Method, system and computer program product for providing a second digital video stream. In a collecting step, a first and a second primary digital video stream are collected from at least two different digital video sources. In a first production step, a first produced video stream is produced based on the first and second primary streams. In a second production step, the second stream is produced based on the first produced stream and also based on the first and second primary streams. In the second production step, the first and second primary streams are time-delayed so as to time-synchronise them with the first produced stream, taking into consideration a latency of the first produced stream resulting from the first production step, the second produced stream being produced based on the time-delayed first and second primary streams.