Core Accelerator Communication Architecture Reducing CPU Overhead
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing on-chip accelerator communication methods suffer from inefficiencies such as slow MMIO write speeds, shared queue management overhead, out-of-order execution, and Quality of Service (QOS) issues, leading to complex software development and high CPU overhead.
Innovation Solution
A universal core to accelerator communication architecture (ACRE) is introduced, which uses a hardware-based approach to unify and simplify the communication interface by employing sending and receiving agents to manage communication channels without software involvement, thereby reducing overhead and complexities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If MMIO read/write operations are used for accelerator communication, then device accessibility is improved, but write speed becomes slow
Solution Approach 1:
The patent introduces a dedicated communication channel with a sending agent and receiving agent as intermediaries between the processor core and accelerator. This mediator structure separates the high-speed data path from the MMIO interface, allowing MMIO to maintain its simplicity while the communication channel handles the speed-critical operations through specialized hardware agents that manage buffering and data transfer efficiently.
Solution Approach 2:
The patent replaces the direct MMIO-based mechanical communication system with a hardware-assisted communication mechanism using sending and receiving agents. These agents use hardware queues and buffers to manage data transfer, substituting the software-based MMIO operations with hardware-managed data paths that achieve higher throughput and faster effective write speeds.
2Adaptability or versatility
If shared queue management is implemented, then communication flexibility is improved, but management overhead increases
Solution Approach 1:
The sending agent and receiving agent autonomously manage their respective queues without requiring software intervention. The sending agent automatically batches requests and manages the send queue, while the receiving agent handles the receive queue and coordinates data transfer. This self-service mechanism provides flexible communication with variable batch sizes while eliminating the CPU overhead associated with software-based queue management.
Solution Approach 2:
The hardware-based sending and receiving agents act as intermediaries that absorb the complexity of queue management. These agents maintain the shared queue structure and handle batch size variations, head/tail updates, and synchronization logic internally, presenting a simplified interface to both the processor core and accelerator while bearing the management overhead themselves.
3Productivity
If out-of-order execution is allowed, then processing throughput is improved, but Quality of Service (QOS) control becomes difficult
Solution Approach 1:
The receiving agent implements feedback mechanisms by monitoring queue occupancy and sending signals back to the sending agent. This feedback loop allows the system to control batch sizes dynamically and manage QOS parameters while maintaining out-of-order execution. The feedback from the receiving agent enables the sending agent to adjust its batching behavior to ensure QOS requirements are met without sacrificing throughput.
Solution Approach 2:
The communication architecture uses dynamic batch size management where the batch size is not fixed but adapts based on queue occupancy and QOS requirements. The sending and receiving agents dynamically adjust their behavior based on real-time conditions, allowing out-of-order execution to proceed while maintaining QOS control through adaptive rather than static queue management.
4Measurement precision
If cache lines are moved across multiple caches, then data availability is improved, but cache line bouncing occurs causing inefficiency
Solution Approach 1:
The communication channel is segmented into distinct sending and receiving agents with dedicated buffers, separating the data transfer path from the cache coherence domain. This segmentation allows data to be transferred directly between agents without necessarily passing through multiple cache levels, reducing cache line bouncing while maintaining data availability through the buffered communication channel.
Data Source
AI summary
Methods and apparatus relating to a universal core to accelerator communication architecture for enhanced performance and/or programmability are described. In an embodiment, a sending agent is coupled to a processor core and a receiving agent is coupled to a hardware accelerator device. Memory store data corresponding to a request from the processor core. The sending agent and the receiving agent maintain a communication channel to facilitate communication between the processor core and the hardware accelerator device in response to the request. Other embodiments are also disclosed and claimed.