Coherence Circuitry for Memory Domain Access

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

In computing systems with multiple processors and cache memories, maintaining cache coherence across different memory domains is challenging, especially when accessing non-coherent memory regions, leading to potential use of stale data and increased software complexity.

Innovation Solution

A system is implemented where circuitry generates commands to access a non-coherent memory domain, translating requests and responses to maintain coherence between coherent and non-coherent memory domains, reducing software overhead and ensuring data consistency across caches.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If writeback cache is used to reduce memory access latency, then cache write performance is improved, but data coherency between cache and main memory deteriorates

Engineering Contradiction:
Improvecache write speedVSAvoiddata coherency
Core Design Contradiction:
SpeedVSReliability

Solution Approach 1:

The patent implements a feedback mechanism where the coherence circuitry monitors cache write operations and generates appropriate memory maintenance commands (such as invalidate or update commands) to keep main memory and other caches synchronized. This feedback loop ensures that when data is written to cache, the coherence system detects this change and propagates the update or invalidation information to other caches and main memory, thus maintaining data coherency while allowing fast writeback cache operations.

Inventive Principle:
Principle #23Feedback

Solution Approach 2:

The patent introduces coherence circuitry as an intermediary component between the cache system and main memory. This intermediary monitors cache operations, translates cache writes into appropriate memory maintenance commands, and coordinates updates across multiple caches and main memory. The intermediary handles the complexity of coherency management separately, allowing the cache to operate at high speed while the intermediary ensures data consistency across the system.

Inventive Principle:
Principle #24Intermediary (Mediator)

2Productivity

If multiple processors access multiple caches simultaneously, then processing parallelism is improved, but cache coherency management complexity increases

Engineering Contradiction:
Improveprocessing parallelismVSAvoidcoherency management complexity
Core Design Contradiction:
ProductivityVSDevice complexity

Solution Approach 1:

The patent segments the coherency management function into a dedicated coherence circuitry component separate from the processors and caches. Each processor and cache can operate independently in parallel, while the coherence circuitry is divided into multiple coherence domains that can handle coherency requests from different processors simultaneously. This segmentation allows parallel processing to proceed without bottlenecking on coherency management, as the coherence system is also parallelized and distributed.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The coherence circuitry acts as an intermediary that mediates between multiple processors and caches, handling coherency protocols without requiring the processors themselves to manage coherency complexity. The intermediary translates processor memory requests into cache-coherent operations, managing the complex state transitions and protocol handshaking required for multi-processor coherency while keeping the processor logic simple and focused on computation.

Inventive Principle:
Principle #24Intermediary (Mediator)

3Adaptability or versatility

If software manages cache coherency manually, then flexibility is improved, but software overhead and complexity increase

Engineering Contradiction:
Improvecoherency management flexibilityVSAvoidsoftware complexity
Core Design Contradiction:
Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent implements self-service coherency management where the hardware coherence circuitry automatically monitors, detects, and resolves cache coherency issues without requiring software intervention. The coherence system autonomously generates maintenance commands, tracks cache state, and ensures data consistency, freeing software from the burden of manual coherency management while maintaining the flexibility of hardware-controlled coherency protocols.

Inventive Principle:
Principle #25Self-service

Data Source

PatentUS10402326B1Accessing memories in coherent and non-coherent domains in a computing system
Publication Date: 2019.09.03 APPLE INC
  • US10402326B1 patent drawing
  • US10402326B1 patent drawing
  • US10402326B1 patent drawing

AI summary

A system that includes circuitry to access memories in both coherent and non-coherent domains is disclosed. The circuitry may receive a command to access a memory included in the coherent domain and generate one or more commands to access a memory in the non-coherent domain dependent upon the received command. The circuitry may send the generated one or more commands to the memory in the non-coherent domain via communication bus.