Accelerator Offloading Data Processing Tasks to Reduce Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems face latency issues during big data analysis due to the co-processor's reliance on local memory, leading to performance bottlenecks when transferring data between the host processor and main memory, which hinders the utilization of accelerator performance.
Innovation Solution
A computing system is designed with a processor, a storage device connected via a network, and an accelerator that offloads data processing tasks, allowing the accelerator to read data directly from storage and execute processes independently, reducing latency by bypassing the need for extensive data transfer between the host processor and main memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If the co-processor reads data from local memory within the same node, then the processing speed is improved, but the local memory capacity runs short and latency increases when reading from external storage
Solution Approach 1:
The patent introduces a host processor as an intermediary that manages data transfer between storage and accelerators. The host processor reads data from storage into main memory, then transfers to accelerator local memory, coordinating the flow to optimize both speed and latency across the system.
Solution Approach 2:
The patent extends the memory hierarchy by adding network-attached storage as an external dimension. Instead of being limited to local memory within the node, the system accesses storage resources across the network, effectively adding a new spatial dimension for data access that increases capacity while managing latency through coordinated transfer.
2Reliability
If data is transferred between host processor and main memory before reaching the accelerator, then data availability is ensured, but processing latency increases
Solution Approach 1:
The host processor performs preliminary actions by pre-loading data into main memory and preparing transfer queues before the accelerator needs the data. This preliminary preparation ensures data availability when the accelerator is ready, reducing waiting time and latency during actual processing.
Solution Approach 2:
The system maintains continuous data flow through pipelined operations where the host processor continuously transfers data from storage to main memory to accelerator local memory. This continuous action eliminates idle waiting periods, ensuring both data availability and minimized latency throughout the processing chain.
Data Source
AI summary
A storage device, connected to a computer including a processor and first memory, and executing a program, stores data processed under the program. The computer includes a protocol processing unit that accesses data in the storage device, an accelerator that includes an arithmetic unit executing a part of a process of the program, and a second memory storing data, and executes the part of the process. The first memory receives a processing request for processing data, and causes the accelerator to execute a command to process data, corresponding to the processing request for the processing request including a process to be executed by the arithmetic unit. The accelerator requests the protocol processing unit to provide target data indicated by a command received from the program, reads data from the storage device via the protocol processing unit, and stores the data in the second memory. The arithmetic unit executes the command.