Hierarchical Arbitration for Memory Page Miss Penalties
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern computing systems face challenges in achieving low latency memory access due to high page miss penalties in high-speed and high-density memory systems like LPDDR4 and LPDDR5, which result in significant bandwidth loss and complexity in arbitration logic, making it difficult to fully utilize the advantages of these memory devices.
Innovation Solution
A hierarchical arbitration structure is implemented, comprising bank set arbitration circuits and a die arbitration circuit, which queues bank data and die data to maintain client request order, prioritizes commands, and manages inter-bank and inter-die timings, enabling aggressive page management and seamless data transfer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a single arbitration logic is used to manage multiple memory banks, then the memory controller can service requests to avoid coherency issues, but the arbitration logic becomes complicated and difficult to meet timing requirements in high frequency systems
Solution Approach 1:
The patent divides the single arbitration logic into multiple hierarchical arbitration circuits (bank-level arbitration circuits and die-level arbitration circuits). Each bank-level arbitration circuit manages arbitration for a specific bank, while die-level arbitration circuits coordinate between different dies. This segmentation reduces the complexity of individual arbitration units while maintaining overall system reliability and coherency through coordinated hierarchical control.
2Productivity
If the number of banks in memory is increased to provide more external memory bandwidth, then bandwidth capacity increases, but page miss penalty increases and bandwidth loss occurs without proper bank management
Solution Approach 1:
The hierarchical arbitration structure performs preliminary actions by predicting future memory access patterns and proactively managing bank states. The arbitration circuits use queuing mechanisms to anticipate page miss scenarios and prepare bank transitions in advance, allowing the system to maintain high bandwidth utilization while minimizing actual page miss penalties through preemptive bank management.
Solution Approach 2:
The patent implements dynamic bank management where the arbitration logic adaptively adjusts bank allocation and access patterns based on real-time workload characteristics. The hierarchical arbitration circuits dynamically select which banks to activate, when to perform page misses, and how to distribute access across multiple banks, optimizing the balance between bandwidth utilization and page miss penalty based on current system conditions.
3Productivity
If aggressive page management is implemented to hide page miss time, then seamless data transfer can be sustained, but the arbitration logic complexity increases
Solution Approach 1:
The patent adds a hierarchical dimension to the arbitration structure by introducing multiple levels of arbitration (bank-level and die-level). This dimensional expansion allows aggressive page management to be implemented at the bank level while die-level arbitration coordinates overall resource allocation, distributing the complexity management across hierarchical layers rather than concentrating it in a single complex unit.
Data Source
AI summary
An apparatus including a plurality of set arbitration circuits and a die arbitration circuit. The set arbitration circuits may each be configured to receive first commands and second commands and comprise a bank circuit configured to queue bank data in response to client requests and a set arbitration logic configured to queue the second commands in response to the bank data. The die arbitration circuit may be configured to receive the commands from the set arbitration circuits and comprise a die-bank circuit configured to queue die data in response to the client requests and a die arbitration logic configured to queue the second commands in response to the die data. Queuing the bank data and the die data for the second commands may maintain an order of the client requests and prioritize the first commands corresponding to a current controller over the first commands corresponding to a non-current controller.