Composite Video Stream Creation Using Flexible Macroblock Ordering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video transmission technologies face challenges in efficiently combining multiple video streams with different resolutions, aspect ratios, and profiles into a single stream while maintaining low latency and compatibility with existing H.264/AVC standards, especially in error-prone environments with limited bandwidth.
Innovation Solution
The use of Flexible Macroblock Ordering (FMO) and Arbitrary Slice Ordering (ASO) techniques allows for the combination of multiple video streams into a single composite stream without decoding or re-encoding, enabling visually synchronized presentation and maintaining compatibility with H.264/AVC standards by restructuring macroblock ordering and adding pad slice groups for rectangular formatting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple video streams with different resolutions and profiles are combined using traditional methods, then compatibility with existing decoders is maintained, but latency increases and bandwidth efficiency decreases
Solution Approach 1:
The patent divides the composite video stream into multiple slice groups, where each slice group corresponds to a specific source stream. This segmentation allows the decoder to process different slices independently and in parallel, eliminating the need to wait for complete frames from all sources, thus reducing composition latency while maintaining H.264/AVC compatibility
Solution Approach 2:
The patent introduces Arbitrary Slice Ordering (ASO) which changes the traditional temporal ordering dimension by allowing slices from different time instances to be interleaved in a single stream. This enables out-of-order delivery and processing of video data, reducing latency while maintaining decoder compatibility through proper slice group organization
2Loss of energy
If video streams are combined without re-encoding, then bandwidth is conserved and latency is reduced, but compatibility with existing H.264/AVC standards becomes compromised
Solution Approach 1:
The patent applies different processing qualities to different parts of the composite stream. Each slice group maintains the original encoding characteristics of its source stream (local quality preservation), while the overall structure conforms to H.264/AVC standards. This allows standard-compatible transmission without full re-encoding, conserving bandwidth
Solution Approach 2:
The patent introduces slice group headers and padding structures as intermediaries that bridge the gap between multiple different video profiles and the standard H.264/AVC format. These intermediary elements enable standard compatibility while allowing the underlying diverse streams to pass through without re-encoding
3Adaptability or versatility
If Flexible Macroblock Ordering is used to create distinct slice groups, then independent stream composition is enabled, but macroblock ordering complexity increases
Solution Approach 1:
The patent implements dynamic slice group organization where the structure and ordering of slices can be flexibly configured based on the specific composition requirements. The slice group headers contain dynamic parameters that allow adaptive organization of macroblocks, enabling versatile stream composition while managing complexity through standardized H.264 syntax
Data Source
AI summary
One system described herein, among others, comprises receiving a plurality of slice groups created by converting a plurality of video stream using flexible macroblock ordering (FMO); compositing a plurality of slice groups to create a sequence of composite pictures, wherein the sequential progression of compositing pictures from the source video streams is according to pictures having a presentation time within a time interval corresponding to the composition operation; adding pad slice groups to the source slice groups to create a composite image compatible with a video standard, wherein the pad slice groups are added to create a rectangular composite picture size compatible with an output display format, and wherein the pad slice groups are keyed with a distinct pattern indicative of padding for the composite pictures; and transmitting a single composite output stream comprised of reordered source slice groups and pad slice groups.