Layered Video Encoding with Skip Frames for High Frame Rates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increased demand for encoding high-resolution and high-frame-rate video data often exceeds the resources available in cameras, particularly in terms of power and chip space, necessitating more efficient encoding methods.
Innovation Solution
A method that combines a base encoder with a Low Complexity Enhancement Video Coding (LCEVC) encoder, using skip frames to distribute the encoding load, allowing frames to be encoded in both layers without increasing resource requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If a base encoder processes all frames at full resolution, then video quality is maintained, but power consumption and chip space requirements increase
Solution Approach 1:
The video encoding process is segmented into two independent layers: base layer and enhancement layer. The base encoder processes only base layer frames at reduced complexity, while a separate LCEVC encoder handles enhancement layer frames. This segmentation allows the system to maintain video quality through the enhancement layer while reducing the power consumption and computational burden on the base encoder.
Solution Approach 2:
The patent introduces a temporal dimension to the encoding process by using skip frames that reference previous frames. Instead of encoding every frame independently at full resolution, the system encodes frames at different temporal resolutions, using motion compensation and frame differencing to represent intermediate frames. This dimensional approach maintains quality while reducing instantaneous processing requirements.
2Productivity
If image resolution and frame rate are increased, then video data quantity increases, but encoding resources become insufficient
Solution Approach 1:
The encoding workload is segmented between two specialized encoders: a base encoder handling base layer frames and an LCEVC encoder handling enhancement layer frames. Each encoder is optimized for its specific function, allowing the system to process high-resolution, high-frame-rate video without requiring a single overly complex encoder. The segmentation distributes computational resources efficiently.
Solution Approach 2:
The patent changes the encoding parameters dynamically based on frame type. Base layer frames are encoded with standard parameters, while enhancement layer frames use different parameters optimized for residual encoding. The system adjusts compression ratios, block sizes, and transformation parameters according to the layer and frame type, enabling efficient processing of high-throughput video data with adaptable resource usage.
3Ease of operation
If a single encoder processes all frames, then encoding is simple to implement, but it cannot handle high frame rates with limited resources
Solution Approach 1:
The patent segments the encoding function into two independent but coordinated encoders with clearly defined roles. The base encoder generates base layer frames that serve as reference for the LCEVC encoder, which then generates enhancement layer frames. This segmentation maintains ease of implementation through modular design while significantly improving frame processing capability by distributing the workload.
Solution Approach 2:
The base encoder serves multiple functions: it encodes base layer frames for display and generates reference frames for the enhancement layer encoding. The LCEVC encoder similarly serves dual purposes by encoding enhancement frames and maintaining temporal relationships. This multi-functionality allows the system to handle high frame rates without requiring completely separate encoding paths for each function.
Data Source
Figure 1~5
Figure 2
Figure 3
AI summary
The present disclosure relates to encoding a sequence of frames in a video stream, comprising receiving the sequence of frames at a first frame rate, encoding every second frame in the received sequence in a first base layer employing intercoding and intracoding, inserting skip frames between the frames encoded in the first base layer, such that every second frame in the first base layer is intercoded with a reference to copy image content of a previous encoded frame in the first base layer, encoding remaining frames in the received sequence in a first Low Complexity Enhancement Video Coding, LCEVC, layer associated with the first base layer, employing residuals and references to corresponding skip frames in the first base layer, and embedding the first LCEVC layer in the first base layer to obtain a first sequence of encoded frames at the first frame rate.