Memory Group Migration for Low-Latency Expanded Virtual Memory
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The separation of CPU and GPU memories by a PCI-Express Bus complicates programming, and using an expanded virtual memory that integrates these memories leads to high software overhead and latency due to demand paging.
Innovation Solution
A host processor generates memory groups within an expanded virtual memory, migrates target memory groups based on memory usage information, and manages these groups using a range search algorithm to reduce latency and overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If CPU memory and GPU memory are physically separated and divided by PCI-Express Bus, then each memory can be optimized for its specific function, but it becomes complicated for programmers to design or write programs
Solution Approach 1:
The patent merges CPU memory and GPU memory into a unified expanded virtual memory space, allowing both memory types to be accessed through a single address space. This integration maintains the physical separation and functional optimization of each memory while providing a unified view to programmers, thereby reducing programming complexity without sacrificing memory-specific performance characteristics
2Ease of operation
If an expanded virtual memory that integrates CPU memory and GPU memory is used, then it becomes easier to design or write programs, but it leads to high software overhead and latency due to demand paging
Solution Approach 1:
The patent implements a memory group management mechanism that proactively manages memory page migration between CPU memory and GPU memory. By predicting memory access patterns and pre-positioning data in appropriate memory locations, the system reduces the need for demand paging operations, thereby lowering software overhead and access latency while maintaining the unified virtual memory space that facilitates easy programming
3Productivity
If demand paging is used in expanded virtual memory, then memory space can be efficiently utilized, but it causes high software overhead and latency
Solution Approach 1:
The patent employs a feedback mechanism through memory group management that monitors memory access patterns and dynamically adjusts memory page placement between CPU and GPU memory. This feedback-driven approach allows the system to maintain high memory space utilization while reducing page fault latency by proactively managing memory pages based on observed usage patterns, thereby optimizing both productivity and response time
Data Source
AI summary
An electronic device includes: a host processor configured to: generate a memory group allocated to an expanded virtual memory in response to receiving memory usage information; determine a target memory group comprising a page on which a page fault has occurred among memory groups allocated to the expanded virtual memory, based on the memory usage information; and migrate the target memory group from one of a host memory, an accelerator memory, and a storage device, which comprises the target memory group, to either one of the host memory and the accelerator memory in which the page fault has occurred; and an accelerator configured to perform an operation using the accelerator memory, wherein the expanded virtual memory is a virtual single address space using the host memory, the accelerator memory, and the storage device.