Distributed Out-of-Order Adaptor Circuits for Memory Interface Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Integrated circuits with a mixture of out-of-order and in-order clients face inefficiencies due to the need for data reordering at the memory controller, leading to sub-optimal latency, higher power consumption, and increased costs, as out-of-order clients' efficiency is compromised by the presence of in-order clients.
Innovation Solution
The implementation of distributed out-of-order adaptor circuits between in-order clients and the memory controller, which perform local data reordering using a command ID generator and reorder buffer, eliminating the need for data reordering at the memory controller and allowing out-of-order scheduling without throttling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data reordering is performed at the memory controller for in-order clients, then in-order clients can receive data in correct sequence, but out-of-order clients' efficiency is compromised and latency increases
Solution Approach 1:
The patent segments the data reordering function from the memory controller and distributes it to individual client adapters. Each adapter handles reordering locally for its associated in-order client, preventing the memory controller from being bottlenecked by reordering operations and allowing out-of-order clients to operate efficiently without being blocked by in-order client requirements.
Solution Approach 2:
The patent moves the reordering operation from the central memory controller dimension to the distributed client adapter dimension. This dimensional shift allows parallel reordering operations at multiple client adapters simultaneously, eliminating the sequential bottleneck at the memory controller and reducing overall latency.
2Adaptability or versatility
If data reordering circuitry is provided at the memory controller for each client, then all clients can be supported, but circuitry complexity and cost increase
Solution Approach 1:
Instead of implementing one large reordering circuit at the memory controller, the patent segments the reordering functionality into multiple smaller, identical client adapter modules. Each adapter handles reordering for one or more in-order clients independently, reducing the complexity burden on the memory controller while maintaining support for multiple client types.
Solution Approach 2:
Each client adapter is self-sufficient with its own reordering buffer and logic, eliminating the need for the memory controller to provide centralized reordering services for all clients. This self-service approach reduces the memory controller's complexity while ensuring each in-order client receives properly ordered data.
3Reliability
If the memory controller always reorders data before returning to clients, then in-order clients are satisfied, but memory bandwidth utilization decreases
Solution Approach 1:
The patent extracts the data reordering operation from the critical memory access path at the memory controller. By moving reordering to client adapters that operate in parallel with memory transactions, the memory controller can focus on high-speed data transfer without being slowed down by reordering operations, thus maintaining high bandwidth utilization while ensuring data correctness.
Solution Approach 2:
The client adapter acts as an intermediary between the memory controller and in-order clients. It receives data in any order from the memory controller and performs local reordering before presenting data to the client, thus decoupling the memory controller's high-speed operation from the client's ordering requirements and maintaining optimal bandwidth utilization.
Data Source
AI summary
An integrated circuit may include memory interface circuitry for communicating with an external or in-package memory module. The integrated circuit may also include out-of-order (OOO) clients and in-order (IO) clients that issue read and write commands to the memory interface circuitry. The memory interface circuitry may include a memory controller having an OOO command scheduler, a write data buffer, and a simple read data pipeline. The memory interface circuitry may also include a multiport arbitration circuit for interfacing with the multiple clients and also OOO adaptor circuits interposed between the multiport arbitration circuit and the IO clients. Each of the OOO adaptor circuits may include an ID generator and a local reordering buffer and may allow the memory controller to return data to the various clients without throttling.