Offloading Tensor Operations to Data Storage Controllers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data storage devices face inefficiencies in performing computational operations due to latency in retrieving data from non-volatile memory and storing results, which increases processing time and resource usage, and existing solutions either increase network bandwidth or require upgrading computing device components.
Innovation Solution
Offloading computational operations, such as tensor operations, to a hot swappable data storage device with integrated computation engines, allowing these operations to be performed directly on the storage device, reducing latency and resource usage on the processing device.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If computational operations are performed by the computing device using data from non-volatile memory, then processing capability is maintained, but latency increases due to data retrieval and storage operations
Solution Approach 1:
The patent extracts computational operations from the computing device and relocates them to the data storage device. The processing device receives requests to perform computational operations, offloads these operations to the data storage device's controller, which executes them using computation engines integrated into the storage device. This separation removes the computational bottleneck from the computing device while maintaining data locality at the storage device, thereby reducing latency and improving operational speed.
2Productivity
If computational operations are offloaded to data storage device, then processing time is reduced, but device complexity increases
Solution Approach 1:
The patent implements multi-functionality in the data storage device by integrating computation engines into the storage device controller. The controller is designed to perform both traditional data storage functions and computational operations (such as tensor operations for machine learning). This universal design allows the storage device to handle multiple tasks without requiring separate dedicated hardware, thereby improving productivity while managing device complexity through consolidated functionality.
3Power
If computational operations are performed at the computing device, then processing capability is sufficient, but network bandwidth increases when results need to be transferred
Solution Approach 1:
The data storage device acts as an intermediary between data storage and computational processing. Instead of the computing device directly processing data and then transferring results over the network, the storage device's controller performs computational operations locally on the stored data. This intermediary role eliminates the need to transfer large datasets between computing and storage devices, reducing network bandwidth consumption while maintaining sufficient computational power through the integrated computation engines.
Data Source
AI summary
Systems and methods for offloading computational operations. In some implementations a method includes determining whether a data storage device coupled to a computing device is capable of performing a set of computational operations. The data storage device may be hot swappable. The method also includes offloading the set of computational operations to the data storage device in response to determining that the data storage device is capable of performing the set of computational operations. The method further includes performing the set of computational operations on the computing device in response to determining the data storage device is not capable of performing the set of computational operations.