Unified Address Space for CPU GPU Memory Paging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computing systems face challenges in efficiently utilizing GPUs for general computation due to constrained programming environments, separate memory systems, scheduling inefficiencies, and power consumption issues, which limit the types of commands that can be executed on GPUs, leading to increased processing burden on CPUs and hampered system performance.
Innovation Solution
A combined CPU/GPU architecture with a unified address space and infrastructure between CPU cores and the I/O Memory Management Unit of the GPU, enabling notification and control information flow without requiring changes to legacy operating systems, allowing for efficient use of the I/O Memory Management Unit and improved resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If separate memory systems are used for CPU and GPU, then each processor can have independent memory access, but memory latency and power consumption increase due to chip-to-chip interfaces
Solution Approach 1:
The patent implements a unified memory space where CPU and GPU share the same physical memory, eliminating the need for separate memory systems. This merging allows both processors to access memory through a common interface, reducing chip-to-chip communication overhead and power consumption while maintaining independent access capabilities through virtual memory management.
2Productivity
If computational commands are sent to GPU for execution, then computational offloading is achieved, but latency increases due to high dispatch overhead and command queuing
Solution Approach 1:
The patent extracts the command dispatch mechanism from the traditional GPU submission model and integrates it directly into the CPU execution flow. By using virtual memory page faults as triggers for GPU command generation, the system eliminates the need for explicit command queuing and reduces dispatch overhead, allowing computational tasks to be offloaded with minimal latency.
3Reliability
If context switching is constrained to coarse granularity, then hardware resource occupation is reduced, but process management flexibility and QoS guarantees are limited
Solution Approach 1:
The patent implements dynamic context switching at fine-grained granularity by leveraging virtual memory page faults. Instead of fixed coarse-grained context switches, the system dynamically triggers context switches whenever a page fault occurs, allowing flexible process management and QoS guarantees while maintaining efficient hardware resource utilization through demand-driven activation of GPU commands.
4Ease of manufacture
If legacy operating systems are used without modification, then system compatibility is maintained, but GPU memory paging and unified address space support are insufficient
Solution Approach 1:
The patent introduces an intermediary layer in the form of a custom virtual memory manager that bridges legacy operating systems and modern GPU requirements. This intermediary handles memory paging, address space unification, and GPU command dispatch transparently to the OS, enabling advanced memory management capabilities while maintaining compatibility with legacy operating systems without requiring core OS modifications.
Data Source
AI summary
In a CPU, the CPU having multiple CPU cores, each core having a first machine specific register, a second machine specific register, and microcode which when executed causes a write notification to be issued to the physical address contained in the second machine specific register; receiving in the first machine specific register of a CPU core, a physical page table/page directory base address, receiving in the second machine specific register of the CPU core, a physical address pointing to a location controlled by the IOMMUv2, determining that a control register of the CPU core has been updated, and responsive to the determination that the control register has been updated, executing microcode in the CPU core that causes a write notification to be issued to the physical address contained in the second machine specific register, wherein the physical address is able to receive writes that affect IOMMUv2 page table invalidations.