Image Processing Cache Prefetching for DRAM Latency Hiding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing electronic devices face inefficiencies in hiding DRAM latency during image processing due to unpredictable DRAM accesses and inefficient speculative prefetching strategies, leading to increased bandwidth consumption and performance degradation.
Innovation Solution
Implementing a buffer memory to accumulate pixel positions and values, allowing parallel processing while waiting for data from main memory, and using a cache memory to store a subset of pixel data, with a synchronization scheme to manage cache memory content and reduce latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If speculative DRAM accesses are used to hide latency, then data fetch operations are performed in advance, but prediction accuracy is poor and bandwidth consumption increases
Solution Approach 1:
The patent implements preliminary action through a buffer memory that accumulates pixel positions in advance before actual data fetch operations are needed. The buffer memory stores requested pixel positions and their corresponding addresses, allowing the system to prepare and manage data access sequences proactively, thereby reducing latency without the bandwidth waste of inaccurate speculative prefetching.
2Volume of stationary object
If fully associative cache approach is used, then cache memory size is reduced, but address matching logic becomes complex and performance decreases
Solution Approach 1:
The patent applies segmentation by dividing the cache memory into multiple sets, each handling a specific portion of the address space. This segmentation reduces the complexity of address matching logic compared to fully associative caching, while still maintaining efficient cache utilization. Each set can be managed independently with simpler replacement policies.
Solution Approach 2:
The buffer memory acts as an intermediary between the cache memory and main memory. It accumulates pixel positions and manages the mapping between cache lines and main memory addresses, simplifying the overall address matching process. The buffer memory mediates the interaction between different memory levels, reducing the complexity burden on the cache memory itself.
3Device complexity
If directly mapped cache approach is used, then cache memory organization is simple, but cache memory size must be large to store full image lines
Solution Approach 1:
The patent applies partial action by storing only the necessary pixel data in the buffer memory rather than entire image lines. This allows the cache memory to be smaller while still providing sufficient buffering capacity. The buffer memory accumulates only the pixel positions and addresses needed for current processing operations, avoiding the excessive storage requirements of directly mapped caching.
4Loss of time
If data pre-fetch is used to hide DRAM latency, then future data accesses are anticipated, but prediction accuracy is insufficient and certitude is not provided
Solution Approach 1:
The patent implements feedback mechanisms where the buffer memory tracks actual data access patterns and adjusts future accumulation operations accordingly. By monitoring which pixel positions are actually accessed and when data is needed, the system refines its prediction and accumulation strategies, improving reliability over time while maintaining latency hiding capabilities.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An electronic device is provided comprising a main memory for storing a current image, comprising a plurality of pixels. A cache memory is configured to store a subset of the 5 plurality of pixels stored in the main memory. A processing circuitry is configured to successively generate a respective pixel value at a plurality of pixel positions of a processed image and to pre-fetch from the main memory the pixel values of the plurality of pixels of the current image to the cache memory. A buffer memory is configured to store a first of the plurality of pixel positions and to accumulate further pixel positions of the plurality of pixel 10 positions, in parallel to the pre-fetching of the pixel values to the cache memory. A corresponding method is also provided.