Dynamic Cache Granularity Partitioning for Mixed CPU-GPU Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional memory modules do not support multiple access granularities, leading to conflicts and increased latency, cost, and reduced space when processing components with different optimal data access granularities, such as CPUs and GPUs, access shared memory resources.
Innovation Solution
A memory sub-system with dynamic access granularity in a cache media, where a cache controller adjusts status bits of cache line metadata based on the access granularity of each cache line, allowing for flexible management of status bits depending on the access granularity, and optionally partitioning the cache memory into separate granularities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If conventional memory modules provide single access granularity, then device complexity is reduced, but data access latency increases when multiple processing components with different granularities access shared memory
Solution Approach 1:
The cache memory is segmented into multiple partitions, each dedicated to a specific access granularity (e.g., 32-byte granularity partition and 64-byte granularity partition). This segmentation allows different processing components to access memory through their respective granularity partitions without interference, eliminating conflicts and reducing latency while maintaining manageable complexity through physical separation.
Solution Approach 2:
The memory subsystem dynamically routes access requests to appropriate granularity partitions based on the access size requirements. The system adapts its behavior by directing 32-byte requests to the fine-granularity partition and 64-byte requests to the coarse-granularity partition, enabling flexible multi-granularity support without requiring complex software management.
2Adaptability or versatility
If additional components are added to manage multiple access granularities, then adaptability improves, but device complexity and cost increase
Solution Approach 1:
Multiple access granularity support is merged into the existing cache memory structure by creating partitions within the same physical memory device. This eliminates the need for separate memory devices or additional management components, as the single memory subsystem handles multiple granularities through internal partitioning, thereby reducing overall system complexity and cost.
Solution Approach 2:
The cache memory partitions are designed to serve multiple functions within a single subsystem. The same memory device can simultaneously serve 32-byte granularity needs and 64-byte granularity needs through its partitions, making the system universally adaptable to different processing components without requiring specialized dedicated memory for each granularity type.
3Adaptability or versatility
If host system processes data segments manually according to access granularity, then adaptability improves, but productivity decreases due to additional processing overhead
Solution Approach 1:
The memory subsystem performs self-service by automatically routing access requests to the appropriate granularity partition based on the request characteristics. This eliminates the need for host system software to manually segment and manage data accesses, as the memory controller handles the granularity management internally, thereby maintaining high productivity while supporting multiple processing components.
Data Source
AI summary
A method comprising receiving a memory access request comprising an address of data to be accessed and determining an access granularity of the data to be accessed based on the address of the data to be accessed. The method further includes, in response to determining that the data to be accessed has a first access granularity, generating first cache line metadata associated with the first access granularity and in response to determining that the data to be accessed has a second access granularity, generating second cache line metadata associated with the second access granularity. The method further includes storing the first cache line metadata and the second cache line metadata in a single cache memory component.