Asymmetric Shared Virtual Memory for CPU-GPU Data Exchange
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current GPU programming models are inefficient due to the inability to directly access data between host and device memory, relying on data arrays and lacking virtual paging mechanisms, making them sensitive to physical memory size and limiting data exchange.
Innovation Solution
The implementation of an Asymmetric Shared Virtual Memory (A-SVM) model, where device memory acts as a 'mega-cache' for host memory, enabling page-level caching and dynamic swapping, and supporting virtual paging by using host memory as a 'swap file' for graphics memory, allowing for efficient data exchange and memory coherency between CPU and GPU.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If data is copied explicitly between host and device memory, then data transfer is possible, but programming efficiency deteriorates and data exchange is limited
Solution Approach 1:
The patent merges host memory and device memory into a unified virtual address space, allowing both CPU and GPU to access the same memory region without explicit data copying. The virtualization layer translates addresses to route data flows appropriately, eliminating the need for programmers to manually copy data between separate memory spaces while maintaining efficient data transfer paths.
Solution Approach 2:
The patent introduces a virtualization layer as an intermediary between host and device memory systems. This layer includes address translation mechanisms and data flow routing logic that automatically manages data transfer, allowing efficient communication without requiring explicit copy operations in the programming model.
2Adaptability or versatility
If physical memory size on graphics card is limited, then device complexity is reduced, but adaptability to different applications deteriorates
Solution Approach 1:
The patent implements a virtual paging mechanism that allows the graphics device to access both local device memory and remote host memory through a unified interface. This multi-functional memory system can dynamically allocate storage capacity between local and remote memory, providing adaptability to different application requirements without requiring separate memory management interfaces.
Solution Approach 2:
The patent employs dynamic memory allocation through virtual paging, where the boundary between local and remote memory is not fixed but can be adjusted based on application needs. The system can dynamically swap data between device memory and host memory, providing flexible memory capacity that adapts to different workloads while managing complexity through standardized paging operations.
3Productivity
If pointers are used for data structures, then data exchange efficiency is improved, but compatibility with current GPU model deteriorates
Solution Approach 1:
The patent combines host and device memory into a unified virtual address space where pointers can naturally reference data structures across both memory regions. This unified model allows complex data structures with pointers to be shared between CPU and GPU without requiring special handling or conversion to index-based access, maintaining both efficiency and compatibility.
Data Source
AI summary
In one embodiment, the present invention includes a device that has a device processor and a device memory. The device can couple to a host with a host processor and host memory. Both of the memories can have page tables to map virtual addresses to physical addresses of the corresponding memory, and the two memories may appear to a user-level application as a single virtual memory space. Other embodiments are described and claimed.