Memory Fetch Granule for Real-Time Agents
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Memory controllers in computing systems face challenges in efficiently utilizing bandwidth, particularly in distributed memory subsystems with multiple real-time agents, leading to interference and head of line blocking, which reduces available memory bandwidth and increases latency.
Innovation Solution
Implementing a memory fetch granule, where real-time agents group their memory requests into larger chunks based on the worst-case memory bandwidth requirements, ensuring these requests are arbitrated as a group to maintain high channel throughput and prevent interference.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple real-time agents access memory channels independently, then each agent can operate autonomously, but head of line blocking occurs and memory bandwidth efficiency decreases
Solution Approach 1:
The patent merges multiple individual memory requests from different real-time agents into a single grouped request (memory fetch granule) that is arbitrated together. This combining approach maintains agent autonomy while preventing head of line blocking by ensuring that grouped requests are processed as a unit, thereby improving memory bandwidth efficiency.
Solution Approach 2:
The patent performs preliminary grouping of memory requests at the agent level before they reach the arbitration point. By pre-grouping requests into memory fetch granules based on worst-case bandwidth requirements, the system prepares requests in advance to be arbitrated as a unit, eliminating the need for complex arbitration decisions and preventing blocking.
2Speed
If memory requests are processed individually, then arbitration can be simple and fast, but bandwidth is not efficiently utilized due to interference and blocking
Solution Approach 1:
The patent combines multiple individual requests into a single memory fetch granule that is arbitrated as one unit. This merging maintains fast arbitration by reducing the number of arbitration decisions needed, while simultaneously improving bandwidth utilization by ensuring efficient use of memory channels through grouped processing.
Solution Approach 2:
The patent segments the memory access process into distinct phases: request generation at agent level, grouping into memory fetch granules, arbitration of grouped requests, and execution at memory level. This segmentation allows simple fast arbitration at the group level while maintaining efficient bandwidth utilization through structured processing.
3Adaptability or versatility
If memory access granularity is small (cache line size), then memory access flexibility is high, but more arbitration decisions are needed reducing bandwidth efficiency
Solution Approach 1:
The patent segments memory access into two levels: fine-grained segmentation at the request level (maintaining flexibility) and coarse-grained segmentation at the memory fetch granule level (improving throughput). By grouping multiple fine-grained requests into larger fetch granules, the system maintains access flexibility while reducing the number of arbitration decisions needed.
Solution Approach 2:
The patent introduces a new dimension of granularity (memory fetch granule) that operates above the traditional cache line level. This additional dimensional layer allows the system to maintain fine-grained access flexibility for individual requests while achieving coarse-grained efficiency through grouped arbitration, effectively resolving the throughput-flexibility tradeoff.
Data Source
AI summary
Systems, apparatuses, and methods for implementing a memory fetch granule for real-time agents are described. A computing system includes a plurality of real-time agents coupled to memory via an interconnect fabric and a memory controller. The efficiency of the memory controller is determined by the number of bank groups in the memory devices coupled to the memory controller. A memory fetch granule is defined for the memory controller based on the amount of data that can be accessed in parallel on the memory device in back-to-back access cycles. Each real-time agent accumulates memory requests for sequential physical addresses until the amount of data referenced by the requests reaches the size of the memory fetch granule. Once the memory fetch granule is reached, the real-time agent sends the requests to the memory controller via the fabric. This helps to ensure that the requests will arrive at the memory controller near enough to each other to get grouped together.