Multi-Die Cache Injection for Lower Inter-Die Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Multi-die processing systems experience increased latency due to communication and data transfer delays between semiconductor dies, impacting performance and efficiency.
Innovation Solution
Implementing a cache injection mechanism with a history buffer and source cache injection logic to proactively populate cache lines at destination processors, reducing cache misses and latency by asynchronously transferring cache lines from source to destination processors using triggers like IPIs or spin unlock operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data is transferred between semiconductor dies in a multi-die processing system, then functionality and performance are enhanced, but latency increases due to communication and data transfer delays
Solution Approach 1:
The patent implements a cache injection mechanism that proactively transfers cache lines from source die to destination die before they are actually needed by the destination processor. This preliminary action populates the destination cache in advance, so when the destination processor needs the data, it is already available in the local cache, eliminating the latency of inter-die data transfer while maintaining the functionality benefits of multi-die architecture
2Reliability
If cache lines are transferred between dies, then cache misses are reduced, but data transfer delays increase
Solution Approach 1:
The system performs cache line transfers in advance based on predicted future needs, transferring data during periods when the destination processor is not yet requesting it. This timing strategy ensures high cache hit rates while minimizing the impact on critical data access paths, as the transfers occur proactively rather than reactively during actual data needs
Solution Approach 2:
The patent introduces an inter-die cache injection mechanism that acts as an intermediary between the source and destination dies. This mechanism includes injection logic at both source and destination dies, along with tracking structures like history buffers and lock wait queues, that manage and optimize the cache line transfers to reduce overall system latency
3Ease of operation
If inter-processor interrupts are used for communication, then processor coordination is improved, but IPI processing time increases
Solution Approach 1:
The system uses IPIs to trigger cache injection operations in advance. When source and destination processor cores are about to execute code that will require data exchange, an IPI is sent to pre-load the necessary cache lines into the destination cache before the actual data access occurs, reducing the critical path latency
Solution Approach 2:
The patent implements tracking mechanisms including history buffers that record recent lock operations and lock wait queues that identify the next processor to acquire a lock. This feedback information is used to intelligently determine which cache lines should be injected and when, optimizing the timing and target of cache injection operations to minimize IPI processing overhead
Data Source
AI summary
According to one aspect, a system includes a processor core, a history buffer, a history buffer logic, and an interrupt controller. The processor core is configured to issue a trigger that causes the history buffer logic to, after a fixed time, flush existing entries in the history buffer and start adding new entries into the history buffer. According to another aspect, a system includes a processor core that includes a load store tracker buffer and a prefetch engine. The buffer is configured to track a critical section memory location associated with critical section data fetched by the processor core. The prefetch engine is configured to obtain the critical section memory location from a previous load store tracker buffer associated with a previous lock holder processor core. The system also includes a lock manager configured to signal the buffer to start and stop tracking the critical section memory.