Interleaved Cache Lines for Multi-Thread Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cache management systems struggle to maintain high cache hit rates when multiple execution threads access different locations in main memory, reducing spatial locality and increasing average access time.
Innovation Solution
Interleaving memory access sequences for multiple execution threads by organizing sequences of consecutive address sub-ranges in an interleaved manner, allowing data values associated with different threads to be stored in a single cache line, thereby increasing the likelihood of cache hits.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data from multiple execution threads are stored in separate cache lines, then each thread has dedicated cache storage, but cache hit rate decreases when threads access different memory locations
Solution Approach 1:
The patent merges data from multiple execution threads into a single cache line by interleaving their address sequences. Instead of allocating separate cache lines for each thread, the system combines data values from different threads (Thread 0, Thread 1, Thread 2, Thread 3) into one cache line, allowing the cache to serve multiple threads simultaneously and increase overall cache hit rate.
Solution Approach 2:
The patent segments the cache line into multiple sub-ranges, each sub-range storing data from a specific execution thread. The cache line is divided into segments (e.g., sub-range 0 for Thread 0, sub-range 1 for Thread 1, etc.), allowing each thread's data to be identified and retrieved based on address segmentation while sharing the same physical cache line.
2Loss of time
If cache memory stores copies of frequently accessed data, then average access time decreases, but memory capacity required increases
Solution Approach 1:
By merging data from multiple threads into a single cache line, the patent effectively increases the amount of useful data stored per unit of cache capacity. This reduces the total memory capacity needed while maintaining high cache hit rates, as the same physical cache space serves multiple threads concurrently.
Solution Approach 2:
The cache line becomes universal by serving multiple execution threads simultaneously. A single cache line can contain data for Thread 0, Thread 1, Thread 2, and Thread 3, making the cache structure multi-functional and reducing the overall cache size requirement while maintaining fast access times.
3Reliability
If spatial locality is assumed for caching consecutive memory locations, then cache hit likelihood increases for sequential access, but performance degrades when multiple threads access different locations concurrently
Solution Approach 1:
The patent segments the cache line into thread-specific sub-ranges, allowing each thread's sequential access pattern to be maintained within its designated segment. Thread 0 accesses sub-range 0, Thread 1 accesses sub-range 1, and so on, preserving spatial locality benefits for each thread while enabling concurrent execution without performance degradation.
Solution Approach 2:
Different parts of the cache line have different qualities assigned to them based on thread association. Each sub-range within the cache line is locally optimized for its corresponding thread's access pattern, allowing spatial locality to benefit each thread individually while the overall cache line serves multiple threads concurrently.
Data Source
Figure 1A
Figure 1B
Figure 1C
AI summary
A method for caching memory, comprising: caching, in a cache line of a cache memory for accessing a physical memory area connected to at least one hardware processor, the cache memory comprising a plurality of cache lines each having a cache line amount of bits, at least two data values, where each data value is of one of at least two ranges of application memory addresses, wherein each range of application memory addresses is associated with one of a set of execution threads having an identified order of threads and executed by the at least one hardware processor, by: organizing a plurality of sequences of consecutive address sub-ranges, each sequence associated with one of the set of execution threads and consisting of a consecutive sequence of application memory address sub-ranges of the respective range of application memory addresses associated with the execution thread, each application memory address sub-range having an identified amount of memory bits less than the amount of cache line bits, in an interleaved sequence of address sub-ranges by alternately selecting, for each execution thread in the identified order of threads, a next address sub-range in the respective sequence of address sub-ranges associated therewith; generating a mapping of the interleaved sequence of address sub-ranges to a range of physical memory addresses in order of the interleaved sequence of address sub-ranges; and when an execution thread of the set of execution threads accesses an application memory address of the respective range of application memory addresses associated thereof: storing the at least two data values in one cache line of the plurality of cache lines by accessing the physical memory area.