3D Transform Inter Prediction Video Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding standards face diminishing returns in performance improvement due to increasing complexity in inter prediction, which complicates computational analysis and may require higher memory access, while temporal redundancy in residual pictures remains unexploited.
Innovation Solution
The method involves encoding and decoding video data using 3D rectangular cuboids spanning multiple consecutive pictures, applying 3D transforms to residual cuboids to exploit temporal redundancy, and using efficient inter-prediction models like 3D Translation and 2D to 3D Mapping to reduce coding costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If inter prediction complexity is increased to improve compression performance, then compression ratio is improved, but computational complexity and memory requirements increase
Solution Approach 1:
The patent extends traditional 2D block-based prediction to 3D cuboid-based prediction by incorporating the temporal dimension. Instead of predicting individual 2D blocks independently, the method groups blocks from multiple consecutive frames into 3D cuboids and performs joint prediction, exploiting temporal redundancy across frames to improve compression without proportionally increasing computational complexity.
Solution Approach 2:
The patent merges multiple 2D blocks from consecutive frames into a single 3D cuboid structure for joint processing. By combining spatially adjacent blocks across multiple frames into one predictive unit, the method reduces the total number of prediction operations needed while capturing temporal correlations, thereby improving compression efficiency without linearly increasing computational load.
2Productivity
If multiple reference pictures and backward prediction are used to improve compression, then compression ratio is improved, but memory access requirements increase
Solution Approach 1:
The patent performs preliminary grouping of blocks from multiple frames into 3D cuboids before the actual prediction process. By pre-organizing the data structure to include temporal information from multiple reference frames, the method enables efficient access patterns that reduce random memory access requirements while still utilizing multiple reference pictures for improved compression.
3Productivity
If 3D transform is applied to residual cuboids to exploit temporal redundancy, then compression performance is improved, but processing complexity increases
Solution Approach 1:
The patent applies 3D transform to the entire cuboid structure including temporal dimension, rather than applying 2D transform separately to each frame. This unified 3D transformation exploits temporal redundancy across frames within the cuboid, achieving better compression performance by capturing correlations in all three dimensions (width, height, time) simultaneously.
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
Methods and devices for video coding using inter prediction and three-dimensional transform. A sequence of pictures in the video is partitioned into coding cuboids. A predicted cuboid is generated for one of the coding cuboids using interprediction, including at least one motion vector and at least one reference picture index. The difference between the coding cuboid and the predicted cuboid results in a cuboid of residuals, which is then spectrally transformed using a three-dimensional transform to create a cuboid of transform domain coefficients, which are then entropy encoded. The decoding process generates the same predicted cuboid and reconstructs the cuboid of pixels using the predicted cuboid and a cuboid of reconstructed residuals.