Non-coherent Store Instruction for L1 Cache Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current high-performance processors with write-through L1 caches incur significant inter-thread communication latency due to the need to update the L2 cache before the L1 cache, which hampers efficient data transfer between threads.
Innovation Solution
A non-coherent store instruction is introduced that allows data to be written directly to the L1 cache before updating the L2 cache, reducing latency by making the data visible to threads sharing the L1 cache without guaranteeing coherence with other memory operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is written to L2 cache before L1 cache to maintain coherence, then data consistency is improved, but inter-thread communication latency increases
Solution Approach 1:
The patent segments the cache update process into two distinct paths: a coherent path that updates both L2 and L1 caches maintaining full consistency, and a non-coherent fast path that updates only the L1 cache for immediate visibility. This segmentation allows threads to choose the appropriate path based on coherence requirements, resolving the contradiction between consistency and speed.
Solution Approach 2:
The patent introduces a coherence parameter (coherent vs. non-coherent flag) that changes the behavior of the store instruction. When the non-coherent flag is set, the system changes from the conventional behavior of updating L2 before L1 to updating L1 immediately, thereby adjusting the coherence parameter to prioritize speed over consistency for specific operations.
2Loss of time
If a non-coherent store instruction is used to write directly to L1 cache, then inter-thread communication latency is reduced, but data coherence guarantee is lost
Solution Approach 1:
The patent introduces a new non-coherent store instruction as an intermediary mechanism that bypasses the normal coherence protocol. This instruction acts as a mediator between the storing thread and the L1 cache, allowing direct writes without L2 involvement. The instruction includes explicit semantics that inform other threads about the non-coherent nature of the write, enabling them to handle the data appropriately.
Solution Approach 2:
The patent inverts the conventional cache update order by allowing L1 cache updates to occur before L2 cache updates for non-coherent operations. Instead of the traditional approach where L2 is updated first to ensure coherence, the invention updates L1 first for immediate visibility and accepts that coherence guarantees are relaxed, effectively doing things 'the other way round' for specific scenarios.
3Reliability
If conventional store instructions are used for inter-thread communication, then data coherence is maintained, but communication speed is limited by L2 cache latency
Solution Approach 1:
The patent applies preliminary action by allowing the L1 cache to be updated in advance before the L2 cache is updated, for non-coherent operations. The storing thread performs the L1 write immediately without waiting for L2, and the data becomes visible to other threads sharing the L1 cache right away. This preliminary update to L1 eliminates the waiting period that would otherwise be imposed by L2 cache latency.
Solution Approach 2:
The patent creates a copy of the data in the L1 cache that is independent of the L2 cache update process for non-coherent operations. This copy in L1 provides immediate visibility to threads sharing the cache, while the L2 cache update proceeds separately in the background. The copying mechanism allows fast access without being blocked by the slower L2 update.
Data Source
AI summary
A method is disclosed that uses a non-coherent store instruction to reduce inter-thread communication latency between threads sharing a level one write-through cache. When a thread executes the non-coherent store instruction, the level one cache is immediately updated with the data value. The data value is immediately available to another thread sharing the level-one write-through cache. A computer system having reduced inter-thread communication latency is disclosed. The computer system includes a first plurality of processor cores, each processor core including a second plurality of processing engines sharing a level one write-through cache. The level one caches are connected to a level two cache via a crossbar switch. The computer system further implements a non-coherent store instruction that updates a data value in the level one cache prior to updating the corresponding data value in the level two cache.