Hierarchical Video Encoding Data Reuse

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In video encoding, the independent processing by multiple software encoders results in significant redundant processing, limiting the number of simultaneous video encodings due to computational constraints, leading to inefficient production of multiple unique video streams with different formats, bit rates, and frame rates.

Innovation Solution

The method involves temporally sub-sampling video frames, converting color formats, performing full motion estimation on one sequence, and reusing motion estimation data for partial motion estimation on subsequent sequences, while adapting hierarchical Human Visual System-based pre-processing to optimize bit rates and remove high-frequency information, thereby reducing redundant processing and enhancing efficiency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If multiple software encoders perform independent and separate encoding processes on the same raw video input, then each encoder can produce video streams with different formats, resolutions, and bit rates, but the processing power available per encoding station limits the number of simultaneous encodings

Engineering Contradiction:

Improveability to produce multiple unique video streamsVSAvoidnumber of simultaneous video encodings

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The encoding process is divided into separate functional modules: motion estimation, DCT transformation, quantization, and entropy encoding. Multiple encoders share common modules (motion estimation, color conversion, temporal sub-sampling) while maintaining independent modules for format-specific processing. This segmentation allows computational resources to be shared across multiple encoding tasks simultaneously.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Multiple encoders are merged into a single encoding station that shares common computational resources. The system combines multiple encoding functions into unified modules that serve multiple encoders, including shared motion estimation engines, color conversion units, and temporal sub-sampling processors, thereby increasing the number of simultaneous encodings possible within available processing power.

Inventive Principle:

Principle #5Merging (Combining)

2Manufacturing precision

If each encoder performs its own independent motion estimation and encoding processes, then each video stream can be optimized for its specific format and bit rate, but significant redundant processing occurs that wastes processor cycles

Engineering Contradiction:

Improveencoding optimization for specific formatsVSAvoidprocessor cycles wasted in redundant processing

Core Design Contradiction:

Manufacturing precisionVSLoss of energy

Solution Approach 1:

Motion estimation is performed once as a preliminary step before encoding multiple video streams. The motion vectors and motion compensation data generated in this initial step are reused by multiple encoders producing different format streams, eliminating the need for each encoder to independently perform the same computationally expensive motion estimation calculations.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system creates copies of motion estimation data and intermediate processing results that can be reused across multiple encoders. Instead of each encoder independently calculating motion vectors, the system generates motion estimation data once and distributes copies to multiple encoders, which then use these copies as input for their format-specific encoding processes.

Inventive Principle:

Principle #26Copying

3Reliability

If the system performs full motion estimation on all video sequences, then accurate motion compensation is achieved for all streams, but the computational bandwidth and encoding time increase significantly

Engineering Contradiction:

Improveaccuracy of motion compensationVSAvoidencoding time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system performs full motion estimation only on select reference sequences that serve as bases for other encodings. For non-reference sequences, the system performs partial motion estimation by reusing and adapting motion data from reference sequences, performing only the additional processing needed to account for format differences. This selective approach maintains adequate motion compensation accuracy while significantly reducing total encoding time.

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS7809061B1Method and system for hierarchical data reuse to improve efficiency in the encoding of unique multiple video streams

Publication Date: 2010.10.05 ADAPTIVE STREAMING INC

AI summary

Data is shared across several simultaneous software video encode/compressions sessions with a common input video source. This reduces the amount of work done by each individual encoder by intelligently sharing/reusing data between encoders at various stages of encoding. Each encoder may use the data from another encoder “as-is” or may further minimally process the data. This data sharing reduces the overall encoding time of the streams significantly by reducing the duplicative work that would otherwise have to be performed by each encoder. Data sharing thus enables production of more simultaneous unique compressed video streams per encoding station. To allow sharing and reuse of data between encoders, various modules in the encoding pipeline operate in a hierarchical manner and use each other's output data, and cooperate to produce multiple unique video streams that may differ in format, bit rate, resolution, frame rate, or other video characteristic.