Die-Stacked Memory QoS Manager for Bandwidth Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional processing systems face inefficiencies due to low memory bandwidth and high memory access latency, particularly during high utilization, which complicates the design and impacts overall processing efficiency, as they rely on complex coordination among devices sharing system memory.
Innovation Solution
A die-stacked memory device with an integrated hardware Quality of Service (QoS) manager that provides centralized QoS functionality, managing memory access requests and resources to achieve specified QoS objectives by tracking metadata and enforcing access policies, thereby reducing the need for complex coordination among devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If system memory is shared among multiple devices, then memory resource utilization is improved, but memory bandwidth and access latency deteriorate during high utilization
Solution Approach 1:
The patent segments the shared memory system into multiple dedicated memory devices, each served by its own memory controller. This segmentation allows parallel access to different memory segments, thereby maintaining high memory bandwidth and low latency even when multiple devices are utilizing memory resources simultaneously.
Solution Approach 2:
The patent introduces memory controllers as intermediary components between external devices and memory devices. These controllers manage and coordinate memory access requests, implementing quality of service (QoS) policies to prioritize certain devices or operations, thus maintaining performance during high utilization without requiring complex coordination among all devices.
2Reliability
If conventional coordination techniques are used for quality of service, then QoS levels are provided, but design complexity increases
Solution Approach 1:
The patent employs memory controllers as intermediary components that centralize QoS management functionality. These controllers implement QoS policies locally at the memory subsystem level, providing reliable quality of service guarantees while avoiding the need for complex inter-device coordination protocols and reducing overall system design complexity.
Solution Approach 2:
The memory controllers autonomously manage QoS enforcement by tracking device utilization and prioritizing memory access requests based on configured policies. This self-service capability eliminates the need for external coordination mechanisms, simplifying the overall system design while maintaining reliable QoS levels.
Data Source
AI summary
A die-stacked memory device implements an integrated QoS manager to provide centralized QoS functionality in furtherance of one or more specified QoS objectives for the sharing of the memory resources by other components of the processing system. The die-stacked memory device includes a set of one or more stacked memory dies and one or more logic dies. The logic dies implement hardware logic for a memory controller and the QoS manager. The memory controller is coupleable to one or more devices external to the set of one or more stacked memory dies and operates to service memory access requests from the one or more external devices. The QoS manager comprises logic to perform operations in furtherance of one or more QoS objectives, which may be specified by a user, by an operating system, hypervisor, job management software, or other application being executed, or specified via hardcoded logic or firmware.