Unified Coherency Controller for Host-Accelerator Memory Hierarchy

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Maintaining coherency in shared virtual memory environments across host processors and accelerators is challenging due to power and performance costs, requiring significant software overhead and complexity, especially in systems with multi-level memory hierarchies.

Innovation Solution

A unified controller manages coherency and multi-level memory by tracking page ownership states and initiating page state transitions, using a page granular controller for coarse granularity and a line granular controller with a snoop filter for finer granularity, enabling transparent coherency management without requiring extensive software or hardware investments.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If traditional coherency management methods are used in shared virtual memory environments with accelerators, then coherency can be maintained, but power consumption and performance costs increase significantly

Engineering Contradiction:

Improvecoherency maintenanceVSAvoidpower consumption

Core Design Contradiction:

ReliabilityVSUse of energy by moving object

Solution Approach 1:

The patent extracts the coherency management function from software and relocates it to dedicated hardware components (coherency management unit and snoop filter). This hardware-based approach maintains coherency reliability while significantly reducing power consumption compared to software-based methods, as the hardware operates autonomously without requiring extensive software overhead.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent introduces a coherency management unit as an intermediary component between the accelerator and memory system. This unit includes a snoop filter that intercepts and filters memory access requests, managing coherency in hardware without requiring full software intervention, thereby reducing both power consumption and performance overhead.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If software-based coherency management is implemented, then coherency can be maintained, but software overhead and complexity increase

Engineering Contradiction:

Improvecoherency maintenanceVSAvoidsoftware overhead

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The coherency management unit operates autonomously using hardware-based snoop filtering, eliminating the need for complex software coherency management. The hardware automatically tracks and manages memory access coherency between accelerators and memory, reducing software overhead and complexity while maintaining reliability.

Inventive Principle:

Principle #25Self-service

3Ease of operation

If hardware-based coherency management is implemented, then software overhead is reduced, but hardware complexity increases

Engineering Contradiction:

Improvesoftware programming simplicityVSAvoidhardware complexity

Core Design Contradiction:

Ease of operationVSDevice complexity

Solution Approach 1:

The patent segments the coherency management functionality into a dedicated coherency management unit with a snoop filter, separate from the main accelerator and memory components. This modular hardware design reduces software programming complexity by providing transparent coherency management, while the segmented architecture helps manage hardware complexity through functional separation.

Inventive Principle:

Principle #1Segmentation

4Adaptability or versatility

If multi-level memory hierarchies are used with accelerators, then system capabilities are enhanced, but coherency management challenges increase

Engineering Contradiction:

Improvesystem capabilitiesVSAvoidcoherency management complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The coherency management unit with snoop filter provides universal coherency management across multi-level memory hierarchies and multiple accelerators. This single hardware component handles coherency for various memory types (DRAM, non-volatile memory) and accelerator configurations, enhancing system capabilities while reducing coherency management complexity through a unified approach.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS10599568B2Management of coherent links and multi-level memory

Publication Date: 2020.03.24 INTEL CORP

AI summary

Techniques for managing multi-level memory and coherency using a unified page granular controller can simplify software programming of both file system handling for persistent memory and parallel programming of host and accelerator and enable better software utilization of host processors and accelerators. As part of the management techniques, a line granular controller cooperates with a page granular controller to support both fine grain and coarse grain coherency and maintain overall system inclusion property. In one example, a controller to manage coherency in a system includes a memory data structure and on-die tag cache to store state information to indicate locations of pages in a memory hierarchy and an ownership state for the pages, the ownership state indicating whether the pages are owned by a host processor, owned by an accelerator device, or shared by the host processor and the accelerator device. The controller can also include logic to, in response to a memory access request from the host processor or the accelerator to access a cacheline in a page in a state indicating ownership by a device other than the requesting device, cause the page to transition to a state in which the requesting device owns or shares the page.