Direct Data Transfer Framework for Accelerator Engines
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hardware-based acceleration systems are inefficient when processing large data sets stored on secondary storage devices, as they require transferring data to the CPU's memory before offloading it to accelerator devices like FPGAs, GPUs, or DSPs, leading to suboptimal performance.
Innovation Solution
A framework that enables direct data transfer from secondary storage devices to accelerator engines, such as FPGAs, GPUs, or DSPs, by using a CPU to locate and program a DMA engine to move data directly from secondary storage to the accelerator, bypassing the need for intermediate CPU memory transfer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is transferred through CPU memory before being sent to the accelerator device, then the data can be processed by the accelerator, but the data transfer time and system overhead increase significantly for large data sets
Solution Approach 1:
The patent introduces a DMA engine as an intermediary device that enables direct data transfer from secondary storage to the accelerator device, bypassing the CPU memory bottleneck. The DMA engine acts as a mediator that manages the data transfer process independently, allowing the CPU to focus on computation while the DMA handles data movement, thus resolving the contradiction between processing speed and transfer time.
Solution Approach 2:
The system segments the data transfer function from the CPU by introducing a dedicated DMA engine. This segmentation allows the CPU to offload the data transfer task to the DMA engine, enabling parallel operation where the CPU can prepare processing tasks while the DMA engine simultaneously transfers data from secondary storage to the accelerator device.
2Ease of operation
If data is transferred through CPU memory, then the CPU can manage the data flow, but the CPU overhead and processing latency increase
Solution Approach 1:
The DMA engine serves as an intermediary that handles the time-consuming data transfer operations between secondary storage and the accelerator device. This allows the CPU to maintain data management capability through high-level control while the DMA engine executes the actual data movement, thereby reducing processing latency without sacrificing ease of operation.
3Productivity
If large amounts of data are transferred to CPU memory first, then the accelerator can access the data, but the memory bandwidth and system resources are consumed
Solution Approach 1:
The patent extracts the data transfer function from the traditional CPU memory path and creates a separate direct transfer path from secondary storage to the accelerator device. This extraction eliminates the need to move large amounts of data through CPU memory, thereby reducing memory bandwidth consumption while preserving the accelerator's ability to access and process the data efficiently.
Data Source
AI summary
Systems and methods for offloading computations from a CPU directly to an accelerator engine are disclosed. One embodiment includes determining a function of an application to be offloaded from a CPU to an accelerator engine, locating data within a file necessary to perform the functions, programming a logic of the accelerator engine based on the function to be offloaded, programming a DMA engine to move a copy the data from a secondary storage device to the accelerator engine, and processing the data at the accelerator engine using the programmed logic.