Media Server Graphic Overlay Insertion for Adaptive Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face complexity in overlay processing for multimedia content streamed over the Internet due to varied device types and encoding technologies, limiting the ability of client devices to generate sophisticated overlays.
Innovation Solution
A media server acts as a transcoding intermediary, providing a flexible API-driven framework to insert graphic overlays before or after video scaling, allowing for adaptive streaming and enabling better performance or visual quality depending on the streaming conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If overlays are added before video scaling, then performance is improved because images are added to only one stream, but visual quality deteriorates because a single overlay image must be used for all frame sizes
Solution Approach 1:
The patent segments the overlay processing by creating separate overlay insertion points for different scaling stages. Multiple overlay modules are implemented - one before scaling and another after scaling - allowing different overlay images to be applied to different scaled versions of the video stream. This segmentation resolves the contradiction by enabling both performance optimization (through pre-scaling overlays) and quality optimization (through post-scaling overlays).
Solution Approach 2:
The patent applies preliminary overlay insertion before video scaling occurs. This preliminary action allows overlay images to be embedded in the original high-resolution stream once, rather than being re-processed for each scaled version. This improves performance while the system simultaneously provides the option to add additional overlays after scaling for quality-critical applications.
2Manufacturing precision
If overlays are added after video scaling, then visual quality is improved because specific overlay images can be provided for each frame size, but processing complexity increases because overlays must be added to multiple scaled streams
Solution Approach 1:
The patent introduces an intermediary architecture with multiple overlay modules positioned at different stages of the video processing pipeline. These intermediary modules act as mediators between the video scaling process and the final output, allowing overlay images to be selectively inserted at optimal points in the processing chain. This intermediary structure manages the complexity by providing a systematic framework for handling multiple overlay insertion points.
Solution Approach 2:
The patent applies local quality by allowing different overlay images to be applied to different scaled versions of the video stream. Each scaled stream (e.g., 720p, 480p, 360p) can have its own optimized overlay image selected and inserted. This local quality approach ensures that overlay visuals are optimized for each specific resolution while the systematic module architecture manages the overall processing complexity.
3Ease of operation
If basic overlays are generated by client devices, then device resource requirements are reduced, but overlay functionality is limited due to restricted access to stream information
Solution Approach 1:
The patent enables client devices to perform self-service overlay generation by providing them with the necessary stream information and overlay configuration data through the adaptive streaming protocol. The client device can autonomously select and insert appropriate overlay images based on the stream metadata and its own capabilities, reducing server processing load while maintaining functional versatility.
Solution Approach 2:
The patent implements a universal overlay framework that can operate at multiple levels - server-side and client-side. The same overlay infrastructure supports both complex server-generated overlays and simplified client-generated overlays. This multi-functionality allows the system to adapt to different device capabilities and network conditions, providing rich overlay functionality when possible and basic functionality when resources are constrained.
Data Source
AI summary
A particular method includes receiving, at a media server, a request for a first rendition of a stream. The method also includes generating a portion of the first rendition. Generating the portion of the first rendition includes determining whether one or more first overlay images are enabled and inserting the one or more first overlay images into one or more frames of the portion of the first rendition conditioned on the one or more first overlay images being enabled. The method further includes sending the portion of the first rendition to a computing device.