Server-Side Video Stream Interleaving for Legacy Client Displays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Interactive television systems face challenges in displaying multiple embedded video streams simultaneously due to hardware and software limitations in legacy client devices, making it difficult to offer a compelling user experience and requiring costly upgrades or brute-force encoding that is computationally expensive.
Innovation Solution
The method involves interleaving user-interface frames with video frames from multiple video streams at the server-side, allowing for efficient decoding and display on client devices, using a system with a fragment encoder, compositor, transcoder/scaler, and interleaver to combine and transmit frames as a single compliant video stream, reducing processing overhead and enabling scalability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple video streams are decoded and displayed simultaneously on legacy client devices, then user experience is improved, but hardware complexity and processing power requirements increase beyond available capabilities
Solution Approach 1:
The patent moves the video processing functionality from the client device to the server, transitioning from a client-capability model to a server-capability model. The server performs video decoding, composition, and encoding, while the client only needs to decode a single composite stream and display it. This dimensional shift resolves the contradiction by providing advanced functionality without increasing client device complexity.
2Adaptability or versatility
If the complete system is virtualized and frame buffer contents are encoded frame-by-frame on a remote server, then flexibility and upgradability are improved, but computational cost and processing overhead increase significantly
Solution Approach 1:
The patent segments the video processing task into independent video stream decodings that are composited onto a single frame buffer, rather than virtualizing the entire system. Each video stream is decoded separately by its own decoder, then the decoded frames are composited and encoded once as a single composite stream. This segmentation reduces computational overhead compared to frame-by-frame virtualization while maintaining flexibility.
Solution Approach 2:
The patent merges multiple video streams into a single composite video stream that is transmitted to the client device. By combining the video streams at the server level and transmitting them as one interleaved stream, the system reduces network bandwidth requirements and client-side processing needs while maintaining the ability to deliver multiple video content streams.
3Adaptability or versatility
If brute-force encoding of frame buffer contents is used, then multiple video streams can be displayed, but processing overhead and computational expense increase
Solution Approach 1:
The patent performs video decoding in advance before composition. Video streams are decoded ahead of time into separate decoded streams, then these pre-decoded streams are composited onto the frame buffer and encoded once. This preliminary decoding action avoids the computationally expensive brute-force approach of capturing and re-encoding the entire frame buffer contents, significantly improving processing efficiency.
Data Source
AI summary
A method of displaying video embedded in a user interface is performed at an electronic device such as a server system or client device. The method includes obtaining user-interface frames having a first placeholder for a first video window and obtaining source video frames having a first video stream in the first video window. The source video frames and the user-interface frames are interleaved to form an output video stream, which is provided for decoding and display.