Shared Page Table Architecture for Multi-Process Neural Processors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural processing units (NPUs) face inefficiencies in handling large deep-learning workloads due to separate memory access by multiple processes, limiting the hit rate of translation lookaside buffers when page tables are not shared.
Innovation Solution
A neural processing device with a shared page table architecture that allows multiple processes to access a unified memory management system, using a shared page table to maintain high hit rates and reduce the need for page table replacement during process switching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If separate page tables are used for each process, then process isolation and security are improved, but translation lookaside buffer hit rate deteriorates due to frequent page table replacements
Solution Approach 1:
The patent merges multiple process-specific page tables into a single shared page table structure. The shared page table contains page table entries for multiple processes, allowing the translation lookaside buffer to maintain translations across process switches without replacement, thereby improving hit rate while maintaining process isolation through logical separation of entries.
Solution Approach 2:
The shared page table serves multiple processes simultaneously, making a single data structure universal for address translation across different processes. This multi-functional approach allows the translation lookaside buffer to service multiple processes from the same cached translations, reducing replacement frequency and improving performance.
2Ease of operation
If separate memory areas are accessed by each process, then memory management simplicity is improved, but memory efficiency deteriorates due to redundant page table structures
Solution Approach 1:
The patent combines multiple separate page tables into one shared page table structure that stores page table entries for multiple processes. This merging eliminates redundant storage of identical or similar page table data across processes, improving memory efficiency while maintaining the ability to manage separate memory areas through process-specific entry ranges or permissions.
Data Source
AI summary
A neural processing device and a method for using shared page table thereof are provided. The neural processing device including at least one neural processor, a shared memory shared by the at least one neural processor, and a global interconnection configured to exchange data between the at least one neural processor and the shared memory, comprises at least one processing unit each of which included in each of the at least one neural processor and configured to provide logical addresses, a memory management unit configured to receive and translate the logical addresses into physical addresses, and a physical memory accessible by the physical addresses, wherein the memory management unit comprises a shared page table that has translation information between the logical addresses and the physical addresses and is shared by at least one process with each other.