Missing Store Operation Accelerator for Cache Coherence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In shared memory multiprocessor systems, store operations incur significant delays and performance impacts due to high store miss rates, which are not adequately mitigated by existing techniques such as store prefetching and increased store buffer/queue sizes, especially in commercial applications with high instruction and data footprints.
Innovation Solution
A missing store operation accelerator is introduced, which indicates exclusively owned memory space units without corresponding data, allowing store operations to commit even if they miss in low-latency memory, by maintaining their coherence state in the accelerator, thus reducing or eliminating unnecessary delays.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If store prefetching is used to improve store memory-level parallelism, then the number of overlapping store misses handled in parallel increases, but L2 cache bandwidth consumption increases substantially
Solution Approach 1:
The patent introduces a store buffer as an intermediary structure between the processor and L2 cache. The store buffer captures store operations and their coherence state information, allowing the processor to continue executing instructions without immediately accessing L2 cache. This mediator absorbs the bandwidth pressure by decoupling store operations from immediate L2 cache access, while still enabling progress through buffered store commits.
2Productivity
If the store queue size is increased to reduce processor stalling, then the processor can handle more store operations, but the processor clock frequency is impacted due to CAM structure constraints
Solution Approach 1:
The patent segments the store handling functionality into multiple independent components: a store buffer for holding store operations, a store queue for managing commit order, and a coherence state tracker for memory consistency. This segmentation allows each component to be optimized independently, avoiding the need for a single large CAM structure that would constrain clock frequency while still providing sufficient store handling capacity.
Solution Approach 2:
The patent uses a simplified tagging mechanism that copies only essential coherence state information (valid bit, exclusive bit, shared bit) rather than maintaining full CAM entries. This copying approach reduces the complexity and size requirements of the store buffer and queue structures, enabling larger capacities without proportionally increasing the clock cycle impact.
3Loss of time
If store buffer and store queue sizes are increased to mitigate store miss impact, then more store operations can be buffered, but the structures cannot be enlarged beyond CAM capacity limits without impacting clock frequency
Solution Approach 1:
The patent changes the parameter representation in store buffer and queue entries by using compact coherence state tracking with specific bits (valid bit, exclusive bit, shared bit) instead of traditional full CAM entries. This parameter optimization reduces the space requirements per entry, allowing the structures to be enlarged to accommodate more store operations without proportionally increasing the overall complexity and clock frequency impact.
4Reliability
If in-order commit of stores is enforced to satisfy processor consistency model, then memory consistency is maintained, but all later stores are held up while a store miss is handled
Solution Approach 1:
The patent implements dynamic store commit ordering where stores can be committed out-of-order when their coherence state allows it. The coherence state tracker dynamically updates the state of each stored address, enabling the store queue to identify which stores are safe to commit without violating memory consistency. This dynamic approach allows later stores to proceed independently when earlier stores are stalled on misses, maintaining consistency while improving throughput.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Maintaining a cache of indications of exclusively-owned coherence state for memory space units (e.g., cache line) allows reduction, if not elimination, of delay from missing store operations. In addition, the indications are maintained without corresponding data of the memory space unit, thus allowing representation of a large memory space with a relatively small missing store operation accelerator. With the missing store operation accelerator, a store operation, which misses in low-latency memory (e.g., L1 or L2 cache), proceeds as if the targeted memory space unit resides in the low-latency memory, if indicated in the missing store operation accelerator. When a store operation misses in low-latency memory and hits in the accelerator, a positive acknowledgement is transmitted to the writing processing unit allowing the store operation to proceed. An entry is allocated for the store operation, the store data is written into the allocated entry, and the target of the store operation is requested from memory. When a copy of the data at the requested memory space unit returns, the rest of the allocated entry is updated.