Virtual-Address Cache Coherency in Large On-Chip Memory AI Processors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Maintaining cache coherency among multiple caches in an artificial intelligence processor with large on-chip memory is challenging, leading to performance deterioration, especially when multiple processor cores process different data based on the same instruction configuration.
Innovation Solution
The method involves setting non-overlapping external memory address areas for multiple caches, providing virtual addresses to allow multiple processor cores to access these caches, and using a bus to manage these areas based on memory map information and Translation Lookaside Buffers (TLB) to ensure cache coherency and efficient data access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple caches are installed for respective processor cores to enable parallel processing, then processing capability is improved, but cache coherency maintenance becomes complex and performance deteriorates
Solution Approach 1:
The patent divides the cache memory system into multiple independent caches, each associated with a specific processor core. Each cache has its own address space and data storage, allowing parallel access without requiring complex coherency protocols. The segmentation is achieved by allocating separate cache memories (e.g., cache 0, cache 1, cache 2) that can be independently accessed by different processor cores through dedicated buses.
Solution Approach 2:
The patent introduces a new dimension of address space by allowing processor cores to access different memory regions (on-chip cache vs. off-chip memory) simultaneously through virtual addressing. This enables core 0 to access off-chip memory while core 1 accesses on-chip cache without conflict, effectively adding a spatial dimension to memory access that resolves coherency issues.
2Adaptability or versatility
If virtual addressing is implemented to allow multiple cores to access different data, then adaptability is improved, but address management complexity increases
Solution Approach 1:
The patent introduces a bus controller as an intermediary that manages address translation and routing between processor cores and memory resources. The bus controller receives virtual addresses from processor cores, translates them to physical addresses, and routes accesses to the appropriate cache or off-chip memory. This intermediary handles the complexity of address management centrally, allowing individual cores to use simple virtual addressing without each core needing complex address translation logic.
3Reliability
If non-overlapping address areas are allocated to multiple caches, then cache coherency is maintained, but memory address space utilization decreases
Solution Approach 1:
The patent implements dynamic address allocation where the bus controller can flexibly assign virtual address ranges to different caches based on current workload requirements. Rather than static non-overlapping allocations, the system dynamically adjusts which cache serves which address range, allowing efficient utilization of address space while maintaining coherency through the bus controller's centralized management of address-to-cache mappings.
Data Source
AI summary
Disclosed herein are a method for supporting cache coherency based on virtual addresses for an artificial intelligence processor having large on-chip memory and an apparatus for the same. The method for supporting cache coherency according to an embodiment of the present disclosure includes, by an artificial intelligence processor including multiple processor cores and multiple caches, setting external memory address areas which do not overlap each other for respective multiple caches; and providing virtual addresses with which the multiple processor cores access the multiple caches.