Cache Memory Coalescing Compressed Data for Bandwidth Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Processing systems that store compressed data in memory often face inefficiencies due to mismatched compression and memory atom sizes, leading to suboptimal memory bandwidth utilization and decreased performance.
Innovation Solution
A system with a cache memory that coalesces compressed data by selecting appropriate groups of blocks in memory, storing compressed data adjacent to existing data within physical tiles to maximize bandwidth during read operations, using a compression bit cache to manage storage states and optimize memory access requests.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the compression atom size is individually optimized based on compression technology while the memory atom size is individually optimized based on memory technology, then each component achieves its optimal performance, but the memory bandwidth cannot be fully utilized when the sizes do not match
Solution Approach 1:
The memory is divided into multiple banks, with each bank containing multiple tiles. Each tile is further divided into compression atoms that can be independently managed. This segmentation allows the system to handle different compression atom sizes efficiently while maintaining full memory bandwidth utilization by organizing data in granular, manageable units across the memory hierarchy.
Solution Approach 2:
The patent introduces a multi-dimensional memory organization where data can be accessed along different dimensions (banks, tiles, compression atoms). This dimensional approach allows the system to accommodate mismatched compression and memory atom sizes by navigating through multiple organizational layers, effectively resolving the size mismatch problem without sacrificing bandwidth utilization.
2Productivity
If compressed data is stored in non-contiguous memory locations following traditional allocation, then memory allocation is simplified, but memory bandwidth is wasted due to retrieving meaningless data alongside compressed data
Solution Approach 1:
The patent merges compressed data from multiple compression atoms into contiguous memory locations. By combining adjacent compression atoms that contain valid compressed data, the system creates contiguous blocks that can be transferred efficiently in single memory operations, eliminating the waste of retrieving meaningless data while maintaining allocation manageability through the underlying bank-tile-atom structure.
Solution Approach 2:
The system performs preliminary organization of compressed data during the write operation, coalescing compression atoms into contiguous blocks before read operations occur. This preliminary action ensures that subsequent reads can efficiently retrieve only meaningful compressed data without requiring additional processing or dealing with scattered allocations, thus optimizing bandwidth utilization.
3Productivity
If the memory system retrieves only the exact size of compressed data, then bandwidth is optimized, but this requires complex memory organization to ensure compressed data aligns with memory access boundaries
Solution Approach 1:
The patent implements a nested memory organization where compression atoms are nested within tiles, which are nested within banks. This hierarchical nesting allows the system to retrieve data at multiple granularities, enabling exact-size retrieval of compressed data while managing complexity through the layered structure. Each level of nesting provides a different granularity of control, allowing the system to optimize for bandwidth while maintaining organizational manageability.
Data Source
AI summary
In one embodiment of the present invention a cache unit organizes data stored in an attached memory to optimize accesses to compressed data. In operation, the cache unit introduces a layer of indirection between a physical address associated with a memory access request and groups of blocks in the attached memory. The layer of indirection—virtual tiles—enables the cache unit to selectively store compressed data that would conventionally be stored in separate physical tiles included in a group of blocks in a single physical tile. Because the cache unit stores compressed data associated with multiple physical tiles in a single physical tile and, more specifically, in adjacent locations within the single physical tile, the cache unit coalesces the compressed data into contiguous blocks. Subsequently, upon performing a read operation, the cache unit may retrieve the compressed data conventionally associated with separate physical tiles in a single read operation.