Hybrid Coherency Tracker for Attached Coprocessor Memory
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current coherency tracking mechanisms for attached coprocessors or accelerators, such as those using Compute Express Link (CXL), face challenges in managing coherent memory access across multiple request agents, leading to complexity, increased chip surface area, and memory bandwidth overhead, particularly when handling concurrent cache line access and ownership transfers.
Innovation Solution
A hybrid coherency tracker (HCT) is introduced, cooperatively managed by hardware and software, which provides a fully coherent view of device memory, simplifying the tracking structure and conserving memory bandwidth by using a hardware component to manage ownership transfer and software to handle indirect access modes, eliminating the need for complex eviction and snooping operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a hardware coherency tracker is used to manage cache line ownership, then coherency tracking accuracy is improved, but device complexity and chip surface area increase
Solution Approach 1:
The coherency tracking function is segmented into two distinct components: a lightweight hardware tracker that monitors ownership state transitions, and a software manager that handles indirect access modes and complex eviction scenarios. This segmentation allows each component to focus on specific tasks, reducing overall system complexity while maintaining accurate coherency tracking.
Solution Approach 2:
The software coherency manager acts as an intermediary between the hardware coherency tracker and the memory subsystem. It receives ownership state information from the hardware tracker and manages the indirect access modes, eliminating the need for complex hardware-based snooping and eviction mechanisms while ensuring coherent memory access.
2Reliability
If traditional coherency tracking mechanisms are used, then memory coherence is maintained, but memory bandwidth overhead increases
Solution Approach 1:
The patent extracts the heavy snooping and eviction management functions from the hardware coherency tracker and relocates them to the software coherency manager. The hardware tracker only performs lightweight ownership state tracking, while the software manager handles indirect access modes and eviction decisions, significantly reducing hardware resource consumption and memory bandwidth overhead.
Solution Approach 2:
The system changes the state management approach by introducing an indirect access mode parameter that allows the hardware tracker to maintain simplified ownership state information. The software manager translates this simplified state information into appropriate memory access commands, reducing the amount of data transmitted over the memory bus while maintaining coherence.
3Reliability
If complex eviction and snooping operations are implemented, then coherency tracking completeness is improved, but device complexity increases
Solution Approach 1:
The software coherency manager performs self-service by directly managing eviction decisions and indirect access modes without requiring complex hardware-based snooping operations. The hardware tracker provides basic ownership state information, and the software manager handles the complex eviction logic, reducing hardware complexity while maintaining complete coherency tracking.
Data Source
AI summary
An apparatus and method for hybrid software-hardware coherency. An apparatus comprises one or more processing elements to process data; a memory controller to couple the one or more processing elements to a device memory; an interconnect to couple the one or more processing elements to a host processor memory and to couple a host processor to the device memory; one or more device caches to store cache lines read from the host processor memory and/or the device memory; coherency circuitry to manage an ownership indication for each cache line, the ownership indication to be set to a first value to indicate ownership by the host processor and to be set to a second value to indicate ownership by the processing device, wherein the coherency circuitry is to transfer ownership of a first cache line from the processing device to the host processor by updating the ownership indication from the second value to the first value, the coherency circuitry to provide indirect access to the cache line by the processing device while the ownership indication is set to the first value, the coherency circuitry to maintain the ownership indication at the first value until receiving a request to change the ownership indication.