Locality-Based Data Fabric Routing to Reduce Memory Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current core and cache hierarchies in data processing systems with 3D stacked memories are not optimized for disparate memory access latencies, leading to reduced performance due to unified structures that do not distinguish between local and remote memory requests.
Innovation Solution
Implement a data processing system with a locality-optimized core to memory interface that routes memory access requests directly to local memory channels, bypassing the crossbar switch for faster processing, and uses separate buffers for local and remote requests to prioritize local access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If unified memory request structures are used, then device complexity is reduced, but memory access latency increases for local requests
Solution Approach 1:
The patent segments memory request handling into separate queues: a first queue for local memory requests and a second queue for remote memory requests. This segmentation allows the system to process local and remote requests independently, preventing local requests from being delayed by remote requests while maintaining manageable complexity through structured organization.
Solution Approach 2:
The patent applies local quality by providing differentiated treatment for local versus remote memory requests. Local requests receive priority processing through a dedicated queue and direct routing path, while remote requests are handled through a separate queue and crossbar switch. This localized optimization improves performance for the most common case (local requests) without completely redesigning the entire memory subsystem.
2Loss of time
If separate queues for local and remote requests are implemented, then memory access latency for local requests is reduced, but device complexity increases
Solution Approach 1:
The patent divides the memory request handling system into segmented queues - a first queue specifically for local requests and a second queue for remote requests. This segmentation reduces latency for local requests by isolating them from remote request traffic, while the modular queue structure keeps complexity manageable through clear separation of concerns.
Solution Approach 2:
The system dynamically routes requests to appropriate queues based on their destination (local vs. remote). The memory interface dynamically selects whether to process requests through the first queue with direct access or the second queue with crossbar switching, allowing flexible adaptation to different access patterns without static complexity.
3Adaptability or versatility
If all memory requests are routed through the crossbar switch, then routing flexibility is maintained, but memory access speed decreases
Solution Approach 1:
The patent extracts the crossbar switch from the critical path of local memory requests. Local requests are routed directly from the memory interface to the memory controller, bypassing the crossbar switch entirely. This extraction maintains the crossbar switch's routing flexibility for remote requests while eliminating it as a speed bottleneck for local requests.
Solution Approach 2:
The patent introduces a queue-based intermediary system that mediates between different request types and the routing infrastructure. The first queue serves as an intermediary for local requests, providing direct access without crossbar switching, while the second queue acts as an intermediary for remote requests that require crossbar routing. This intermediary layer enables speed optimization for local requests while preserving routing flexibility through the crossbar switch for remote requests.
Data Source
AI summary
A data processing node includes a processor element and a data fabric circuit. The data fabric circuit is coupled to the processor element and to a local memory element and includes a crossbar switch. The data fabric circuit is operable to bypass the crossbar switch for memory access requests between the processor element and the local memory element.