Tile Stream Encoding for Scalable Video Mosaics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for generating video mosaics, such as those described in Sanchez et al., face scalability issues due to server processing limitations, introduce delays in client-server signaling, and lack flexibility in handling third-party content providers, while also being inefficient in resource usage and prone to single points of failure for privacy and security concerns.
Innovation Solution
A system and method for forming tile streams by encoding media data from composite video streams into independently encoded tiles, allowing for flexible composition and rendering of video mosaics on client devices using HEVC or similar codecs, with metadata determining tile positions and enabling efficient distribution and decoding without the need for re-encoding or spatial decoding dependencies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If video mosaics are generated by decoding, spatially combining and re-encoding video frames on the server, then video mosaic composition is achieved, but resource consumption increases and quality degradation occurs
Solution Approach 1:
The patent segments the video processing workflow by dividing the video frame into multiple independently encoded tiles. Each tile can be encoded separately without requiring full frame decoding and re-encoding, thus reducing server resource consumption while maintaining video mosaic composition capability
Solution Approach 2:
Instead of the conventional approach of decoding-combining-re-encoding, the patent inverts the process by encoding individual tiles separately and then combining them in the encoded domain. This reversal eliminates the need for resource-intensive decoding and re-encoding operations on the server
2Ease of manufacture
If double encoding is performed at video source and server, then video mosaic is created, but video quality degrades
Solution Approach 1:
The patent inverts the conventional encoding workflow by performing encoding at the source for individual tiles and then combining them in the encoded domain rather than decoding and re-encoding. This single-encoding approach preserves video quality while achieving mosaic composition
Solution Approach 2:
The patent uses copying of encoded tile data to create the mosaic composition. Instead of re-encoding, the system copies and rearranges independently encoded tile segments to form the final mosaic, thereby avoiding quality degradation from multiple encoding passes
3Use of energy by moving object
If server processes combine video streams in encoded domain, then resource usage is reduced, but scalability with number of users is poor
Solution Approach 1:
The patent segments both the video content into independent tiles and the user requests into individual tile stream selections. This allows the server to efficiently handle multiple users by serving pre-encoded tile streams rather than processing each user's mosaic request individually, improving both resource efficiency and scalability
Solution Approach 2:
The patent implements preliminary encoding of video content into multiple tile streams before user requests arrive. This advance preparation allows the server to quickly assemble mosaics for multiple users by selecting and combining pre-encoded tiles, thereby improving scalability without increasing real-time processing load
4Adaptability or versatility
If centralized server controls video mosaic composition, then composition flexibility is achieved, but single point of failure risk increases for privacy and security
Solution Approach 1:
The patent segments the video mosaic composition process into independent tile stream selections that can be made at different locations (source or client). This distribution of composition authority eliminates the single point of failure at the server while maintaining personalization capabilities through client-side or source-side composition options
Solution Approach 2:
The patent inverts the centralized control model by enabling composition at the source or client端 rather than requiring server-mediated composition. This inversion distributes trust and control, reducing privacy and security risks associated with centralized server processing while preserving mosaic personalization
Data Source
AI summary
Methods and systems are described for forming one or more tile streams comprising: providing media data of at least one composite video stream to the input of at least one encoder, the media data of said composite video stream representing video frames comprising an image region having one or more subregions, each of said one or more subregions comprising visual content of one of said one or more video streams; providing encoder information to said encoder for configuring said encoder to generate encoded media data representing tiled video frames, a tile in said tiled video frames matching at least one of said one or more subregions; encoding media data of said composite video stream, wherein media data associated with a subregion of a video frame are encoded into independently encoded media data that have no spatial decoding dependency on media data of other subregions in said video frame, said independently encoded media data forming a tile of a tiled video frame, preferably said tile being arranged in said image region on the basis of a tile grid; and, storing said encoded media data as one or more tile streams on a storage medium, preferably on a storage medium of one or more network nodes, a tile stream comprising media data of at least one tile and tile position information for signaling a decoder to decode the media data of said tile stream into tiled video frames comprising a tile at a position as indicated by the tile position information.