Co-processor Direct NVMe Access for GPU Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computing systems require extensive host CPU involvement for graphics scene generation, leading to complex and latency-prone processes for Level Of Detail (LOD) information transfer between the GPU and non-volatile memory storage systems.
Innovation Solution
A GPU-optimized data flow is implemented, allowing direct co-processor access to prestored file system data in a non-volatile memory system, eliminating the need for host CPU involvement in data load requests. This is achieved through a request command queue and cache buffer in the NVMe system, enabling direct memory access and reducing communication overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If host CPU involvement is used for data load requests between GPU and non-volatile memory, then data transfer can be performed, but latency increases and performance decreases
Solution Approach 1:
The patent extracts the host CPU from the data transfer path between GPU and non-volatile memory. By implementing direct GPU access to the NVMe controller through a dedicated command queue and memory-mapped I/O interface, the CPU is removed from the critical data load request path, eliminating CPU involvement overhead and reducing latency for texture and shader data transfers.
Solution Approach 2:
The patent introduces an intermediary command queue structure that enables direct communication between GPU and NVMe controller. The command queue acts as a mediator that allows the GPU to submit read requests directly to the NVMe controller without CPU intervention, while maintaining proper protocol handling and data flow management.
2Ease of operation
If host CPU involvement is required for graphics scene generation, then data processing can be performed, but device complexity increases
Solution Approach 1:
The patent enables the GPU to serve itself by directly accessing non-volatile memory storage without requiring host CPU assistance. The GPU can independently issue read requests to the NVMe controller, manage its own data buffers, and perform graphics scene generation operations autonomously, thereby simplifying the overall system operation.
Solution Approach 2:
The patent implements a universal interface architecture where the NVMe controller can serve both traditional CPU-based hosts and GPU-based direct access operations through the same command queue and data path infrastructure, eliminating the need for separate processing paths and reducing system complexity.
Data Source
AI summary
An apparatus includes a co-processor that provides process operations for a first processor, such as a host processor, that executes one or more applications. The co-processor issues a data load request to a request command queue controlled by a non-volatile memory system, that loads a subset of prestored data from a prestored set of data, such as file data of an application, that is stored in a cache buffer in the non-volatile memory system or system memory, for an operation performed by the co-processor. The co-processor uses the loaded subset of prestored data for the operation. In certain implementations, the co-processor issues the data load request as a direct memory access (DMA) load request for the cache buffer. The prestored set of data includes a prestored set of data from a file storage system. Associated methods are also disclosed.