Distributed Load Store Queue for Out-of-Order Dispatch
Solution Overview
Problem
Current processor architectures face challenges in managing context switches and parallelism, particularly in fine-grained parallelism scenarios, due to limitations in hardware thread support, leading to inefficiencies in load/store queue management and increased overhead in thread synchronization.
Innovation Solution
Implementing a distributed load/store queue system that allows out-of-order dispatch of loads and stores across multiple cores and memory fragments, enabling speculative execution while maintaining single program sequential semantics and ensuring error-free program execution through recovery from speculative errors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If hardware duplicates all architecture state elements for each supported thread, then context switch is eliminated, but area, power and complexity increase
Solution Approach 1:
The architecture state is segmented into two parts: a small fixed number of hardware-supported thread contexts (e.g., 2-4) and a larger number of software-managed contexts. This segmentation allows the hardware to maintain only essential state elements for quick context switches, while software handles the majority of context management, thereby reducing hardware complexity while maintaining fast switching capability.
Solution Approach 2:
A context descriptor structure serves as an intermediary between hardware and software thread contexts. The hardware maintains a limited number of context descriptors, while software manages the full set of thread contexts in memory. This intermediary layer enables efficient context switching by allowing the hardware to quickly switch between a small set of active contexts while software handles the broader context management, reducing both hardware complexity and context switch time.
2Device complexity
If a single centralized load/store queue is used, then implementation is simpler, but deadlock occurs when queue is full and loads/stores cannot be dispatched
Solution Approach 1:
The load/store queue is segmented into multiple distributed queues, each associated with specific memory fragments or cores. This segmentation eliminates the single point of contention in a centralized queue, allowing loads and stores to be dispatched to different queues simultaneously, thereby preventing deadlock while maintaining manageable complexity through modular queue structures.
Solution Approach 2:
The queue structure transitions from a single-dimensional centralized queue to a multi-dimensional distributed queue system organized by memory fragments and cores. This dimensional change allows parallel dispatch paths through different queue dimensions, eliminating the bottleneck and deadlock conditions of the centralized approach while keeping individual queue implementations relatively simple.
3Reliability
If load/store queue size is increased to prevent deadlock, then dispatch reliability improves, but hardware complexity and area increase
Solution Approach 1:
Instead of increasing the size of a single large queue, the system segments the queue capacity across multiple smaller distributed queues. Each distributed queue maintains a manageable size suitable for its associated memory fragment or core, while the collective system provides the necessary dispatch reliability through parallel pathways and reduced contention, thereby achieving high reliability without proportionally increasing overall hardware complexity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An out of order processor. The processor includes a distributed load queue and a distributed store queue that maintain single program sequential semantics while allowing an out of order dispatch of loads and stores across a plurality of cores and memory fragments; wherein the processor allocates other instructions besides loads and stores beyond the actual physical size limitation of the load/store queue; and wherein the other instructions can be dispatched and executed even though intervening loads or stores do not have spaces in the load store queue.