Video Composition via Compressed Domain Motion Compensation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for composing multiple video bitstreams into a single stream, such as in video conferencing and surveillance systems, face challenges due to computational complexity and signal quality degradation, especially when rearranging or displacing video streams within the composition, as they often require transcoding which limits flexibility and increases bandwidth.
Innovation Solution
The approach involves creating a composed video data stream by inserting 'no-output' pictures that reference inherited pictures via motion-compensated temporal prediction, allowing for spatial location changes without leaving the compressed domain, thus avoiding computational complexity and signal degradation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If full transcoding is used to compose multiple video bitstreams, then video composition flexibility is improved, but computational complexity increases significantly
Solution Approach 1:
The patent segments the video composition process into two distinct parts: (1) a compressed domain processing stage that handles motion compensation and prediction data merging, and (2) an uncompressed domain stage that only processes the final output picture. This segmentation allows most operations to occur in the computationally efficient compressed domain while maintaining flexibility through selective uncompressed processing at the output stage.
Solution Approach 2:
The patent introduces motion compensation data and prediction residual data as intermediary elements that bridge the compressed domain processing and the final uncompressed output. These intermediaries carry the essential video information through the composition process without requiring full decoding, enabling flexible composition with reduced computational complexity.
2Adaptability or versatility
If full transcoding is applied to rearrange video streams in composition, then composition freedom is improved, but signal quality degradation increases
Solution Approach 1:
The patent segments the video data processing into compressed domain operations that preserve original signal quality and an uncompressed domain stage limited to final output generation. This segmentation ensures that rearrangement and composition operations occur without repeated encoding/decoding cycles, thereby preventing signal quality degradation while maintaining composition freedom.
Solution Approach 2:
The patent uses copying of motion compensation data and prediction residual data from input bitstreams to construct the output bitstream. This copying approach in the compressed domain allows for flexible rearrangement of video streams without the signal quality loss associated with repeated transcoding operations.
3Device complexity
If compressed domain stitching is used, then computational complexity is reduced, but flexibility in rearranging video streams is limited
Solution Approach 1:
The patent introduces dynamic spatial location parameters that allow the position of video streams within the composed output to be changed without requiring full transcoding. The motion compensation data and prediction residual data can be selectively applied to different spatial locations in the output picture, enabling flexible rearrangement while maintaining computational efficiency through compressed domain processing.
4Adaptability or versatility
If random access points are used for rearrangement in compressed domain, then composition changes are enabled, but bitrate and bandwidth peaks increase
Solution Approach 1:
The patent performs preliminary composition operations in the compressed domain using motion compensation data and prediction residual data before final output generation. This preliminary action allows composition changes to be prepared without requiring random access points in the compressed bitstreams, thereby avoiding the bitrate and bandwidth peaks that would result from frequent intra-picture coding.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Video composition is performed by filling a portion of the composed video data steam by collecting and copying, and another portion added synthetically by referencing, via motion-compensated temporal prediction, the other portion of the composed video data stream, obtained by collecting and copying.