Video Encoding Cache Locality Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing size of reference frames due to higher picture resolution and the performance gap between processing power and memory communication lead to a memory bottleneck in video processing systems, resulting in high off-chip memory traffic and power consumption, especially in mobile applications, where standard memory components impose fixed bandwidth limits.
Innovation Solution
The method involves using a small on-chip buffer to store a subset of a reference image, allowing multiple images to be processed simultaneously, reducing the need for separate off-chip memory accesses by synchronizing data access based on motion vectors and utilizing task-level parallelism with multiple processing units, thereby minimizing off-chip memory traffic and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Volume of stationary object
If reference frames are stored in off-chip memory to accommodate high resolution video, then storage capacity is sufficient, but memory bandwidth and access latency increase
Solution Approach 1:
The patent segments the reference frame storage into two parts: a first buffer memory on-chip storing a subset of reference frame data, and a second buffer memory off-chip storing the complete reference frame data. This segmentation allows frequently accessed reference data to be stored in fast on-chip memory while maintaining sufficient total storage capacity through the larger off-chip memory.
Solution Approach 2:
The patent introduces an intermediary mechanism (the first buffer memory on-chip) between the processing unit and the off-chip memory. This intermediary caches reference frame data that is likely to be accessed, thereby mediating the speed mismatch between fast processing units and slow off-chip memory.
2Reliability
If multiple reference frames are stored in memory, then encoding/decoding quality is maintained, but memory bandwidth consumption increases
Solution Approach 1:
The patent performs preliminary action by pre-loading likely reference frame data into the first buffer memory on-chip before it is needed by the processing unit. This anticipatory caching ensures that reference data is already in fast memory when needed, reducing bandwidth consumption while maintaining encoding/decoding quality.
Solution Approach 2:
The patent changes the parameter of memory location for reference frame data, storing some reference data in on-chip memory (first buffer) and other reference data in off-chip memory (second buffer). This parameter change optimizes the balance between memory bandwidth consumption and encoding/decoding quality.
3Productivity
If on-chip buffer size is increased to store complete reference frames, then off-chip memory access is reduced, but chip area and cost increase
Solution Approach 1:
The patent applies partial action by storing only a subset of reference frame data in the on-chip first buffer memory, rather than storing complete reference frames. This partial caching provides sufficient performance benefit while avoiding the excessive chip area and cost that would result from storing all reference frame data on-chip.
4Productivity
If off-chip memory bandwidth is doubled to increase processing throughput, then memory bottleneck is relieved, but system cost and chip pin count increase
Solution Approach 1:
The patent solves the bandwidth bottleneck not by increasing bandwidth in the same dimension (doubling off-chip memory interfaces), but by changing to another dimension: using the fast on-chip first buffer memory as an intermediate caching layer. This dimensional change avoids increasing system cost and chip pin count while still relieving the memory bottleneck.
Data Source
Figure 1~2
Figure 3
Figure 4~5
AI summary
The invention relates to a method for encoding/decoding a video stream including a plurality of images (A, B, C) in a video processing apparatus having a processing unit (11) coupled to a first memory (12) , further comprising a second memory (13) , comprising the steps: providing a subset of image data stored in the second memory (13) in the first memory (12) ,- simultaneous encoding/decoding of more than one image (B, C) of the video stream, by accessing said subset, wherein the simultaneously encoding/decoding is performed by access sharing to at least one image (A) . For reducing the traffic or the number of access to second memory (13) an apparatus is proposed including: a processing unit (11) for performing an encoding/ decoding process of video data; a first memory (12) coupled to the processing unit (11) for storing image data required for encoding /decoding of video data,- a second memory (13) , wherein the video stream includes a plurality of images (A, B, C) ; the first memory (11) is adapted to store a subset of image data of the second memory (13) in the first memory (12) and the processing unit (11) is adapted to simultaneously encode/decode more than one image (B, C) of the video stream by accessing said subset of image data in the first memory (12) , wherein the simultaneously encoding/decoding is performed by sharing access to at least one image .