GPU Virtual Memory Management via TLB Coherency and Prefetching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge in managing virtual memory for graphics processing units (GPUs) is that they cannot access virtual memory addresses, leading to memory pressure due to the need for objects to be pinned, which restricts memory usage and efficiency, especially in general-purpose computation tasks.
Innovation Solution
A method and apparatus that manage virtual memory by tracking GPU access, setting reference bits in page tables, classifying pages based on GPU activity, evicting pages, handling page faults, and maintaining translation lookaside buffer coherency, allowing GPUs to use virtual memory addresses and optimizing memory usage by sharing page tables with central processing units (CPUs).
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If object memory is pinned to prevent paging out to disk, then GPU can access the object memory, but memory pressure increases and memory usage efficiency decreases
Solution Approach 1:
The patent segments memory management by introducing separate page tables for CPU and GPU, allowing independent management of virtual memory spaces. This enables GPU memory to be separately controlled and paged without affecting CPU memory operations, resolving the contradiction between ensuring GPU access reliability and maintaining memory availability.
Solution Approach 2:
The patent introduces a memory manager as an intermediary that handles page faults and manages the page table mappings between CPU and GPU virtual address spaces. This mediator coordinates memory access requests, allowing GPU memory to be paged out when not in use while maintaining reliable access when needed, thus balancing reliability and availability.
2Adaptability or versatility
If separate page tables are maintained for CPU and GPU, then GPU virtual memory management is enabled, but system complexity increases
Solution Approach 1:
The patent merges the management of CPU and GPU page tables under a unified memory manager that handles both address spaces. While separate page tables are maintained for each processor type, the management logic is consolidated in the memory manager, which processes page faults and updates mappings for both CPU and GPU operations, thereby enabling GPU virtual memory capability without proportionally increasing overall system complexity.
3Quantity of substance
If memory is paged out to disk storage, then memory availability increases, but access speed decreases
Solution Approach 1:
The patent implements pre-fetching mechanisms that anticipate GPU memory access needs and load required pages into physical memory before they are actually accessed. This preliminary action ensures that when the GPU needs memory, the data is already in fast physical memory rather than being paged from disk, thus maintaining high access speed while still allowing memory to be paged out when not needed.
Solution Approach 2:
The patent uses reference bits and access pattern monitoring to provide feedback about GPU memory usage. This feedback mechanism allows the memory manager to identify frequently accessed GPU pages and keep them in physical memory, while less frequently accessed pages can be paged out to disk, thus optimizing the balance between memory availability and access speed based on actual usage patterns.
Data Source
AI summary
A method and apparatus of a device that manages virtual memory for a graphics processing unit is described. In an exemplary embodiment, the device performs translation lookaside buffer coherency for a translation lookaside buffer of the graphics processing unit of the device. In this embodiment, the device receives a request to remove an entry of the translation lookaside buffer of the graphics processing unit, where the device includes a central processing unit and the graphics processing unit. In addition, the entry includes a translation of virtual memory address of a process to a physical memory address of system memory of a central processing unit and the graphics processing unit is executing a compute task of the process. The device locates the entry in the translation lookaside buffer and removes the entry.