Network Attached Storage Array Computation Offloading
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems face challenges in efficiently processing large data sets (big data) due to high computational resource and bandwidth consumption when transferring data from network-attached storage (NAS) devices to application servers, leading to latency and resource inefficiency.
Innovation Solution
Extending data-sharing and file-sharing protocols like NFS and CIFS to enable computations to be performed directly on NAS devices using lightweight virtual machines, allowing for remote procedure calls that specify data operations and perform array-based computations, thereby reducing the need for data transfer and offloading computations from application servers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If data is transferred from NAS devices to application servers for analysis, then computation capability is improved, but bandwidth consumption and latency increase
Solution Approach 1:
Instead of transferring data from storage to computation nodes, the patent inverts the approach by bringing computation to the storage nodes. Virtual machines are executed directly on NAS devices, performing computations on data while it remains stored, thereby eliminating the need for data transfer over the network and reducing bandwidth consumption.
Solution Approach 2:
The patent introduces virtual machines as intermediaries that run on NAS devices. These virtual machines act as a mediator between the stored data and the application servers, enabling computations to be performed on-storage without requiring data to be moved to application servers, thus reducing network bandwidth consumption.
2Power
If data is transferred from NAS devices to application servers for analysis, then computation capability is improved, but latency increases
Solution Approach 1:
The patent implements preliminary action by executing computation virtual machines directly on the NAS devices where data is stored. This allows computations to begin immediately on the stored data without waiting for data transfer to application servers, significantly reducing latency while maintaining computation capability.
3Loss of energy
If NAS devices perform computations using virtual machines, then bandwidth consumption is reduced, but device complexity increases
Solution Approach 1:
The patent uses virtual machine copying/virtualization to enable computation on NAS devices. By creating virtual instances that run on the storage hardware, the system enables complex computational tasks without permanently modifying the physical NAS device architecture, thus managing device complexity through software-based virtualization rather than hardware complexity.
4Power
If computations are performed on application servers, then computation capability is maintained, but resource utilization on storage nodes is reduced
Solution Approach 1:
The patent applies universality by enabling NAS devices to perform multiple functions: both data storage and computation execution. By running virtual machines on storage nodes, the system makes storage devices multi-functional, allowing them to contribute computational resources to the overall system productivity and improving resource utilization on storage nodes.
Data Source
Figure 1
Figure 2~3
Figure 4
AI summary
An instruction from an application server to perform a computation is received at a network-attached storage (NAS) device. The computation uses arrays of data that are stored by the NAS device as inputs. The instruction includes remote procedure calls that identify operations that are included in the computation, including a first remote procedure call that will cause the NAS device to perform a read operation on a first file containing an array of data to be used as an input for the computation, and a second remote procedure call that will cause the NAS device to perform an array operation using the array of data. The operations are executed on the NAS device to produce a result that is stored in a second file in a location in a file system managed by the NAS device and accessible to the application server.