Intelligent Storage Devices Bypassing Server for Scalable Distributed File Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Centralized data storage systems face limitations in scalability and efficiency due to the need for multiple compute nodes and increased bandwidth usage, leading to higher costs and power consumption, as well as limitations on the number of hard disks that can be attached to a single server node.
Innovation Solution
Deploying intelligent storage devices directly connected to the network, with programmable network switches that allow data blocks to be transferred directly from these devices to the requester, bypassing the server, and modifying network routes to simulate the server as the sender.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If multiple compute nodes are added to distribute storage, then storage capacity is improved, but power consumption and cooling costs increase
Solution Approach 1:
The patent extracts the computing function from the storage system by allowing storage devices to send data blocks directly to requesters without server intervention. This removes the need for additional compute nodes to handle data transfer, reducing power consumption while maintaining storage capacity.
Solution Approach 2:
Storage devices are empowered to autonomously send data blocks to requesters without requiring server mediation. This self-service capability eliminates the need for additional compute nodes, thereby reducing power consumption and cooling requirements while preserving storage capacity.
2Quantity of substance
If multiple compute nodes are added to distribute storage, then storage capacity is improved, but bandwidth consumption increases
Solution Approach 1:
The patent extracts the data transfer function from the server and performs it directly at the storage device. This eliminates the need for data to traverse back through the server, reducing network bandwidth consumption while maintaining distributed storage capacity.
Solution Approach 2:
The patent introduces a network switch as an intermediary that enables direct communication between storage devices and requesters. This mediator allows data blocks to be transferred efficiently without consuming excessive bandwidth through the server, thus reducing overall bandwidth consumption while preserving storage capacity.
3Quantity of substance
If more hard disks are attached to a single server, then storage capacity is improved, but the number of disks that can be attached is limited
Solution Approach 1:
The patent segments the storage system into independent storage devices that can autonomously communicate with requesters. This segmentation removes the server's bottleneck on the number of attachable disks, allowing each storage device to be independently connected and managed, thus increasing storage capacity without increasing server node complexity.
Solution Approach 2:
The patent extracts the data transfer function from the server, allowing storage devices to operate independently. This eliminates the server's role as a mandatory intermediary, removing the limitation on the number of disks that can be effectively managed by a single server node and enabling greater storage capacity.
4Productivity
If the server assembles data blocks from multiple peers, then file transfer is completed, but CPU load and power consumption increase
Solution Approach 1:
The patent extracts the data block sending function from the server and implements it directly at the storage device. This eliminates the server's CPU-intensive block assembly operation, reducing CPU load and power consumption while maintaining file transfer completion through direct block delivery to requesters.
Solution Approach 2:
Storage devices autonomously send data blocks directly to requesters without requiring server assembly. This self-service approach eliminates the server's CPU-intensive block assembly operation, reducing CPU load and power consumption while ensuring file transfer completion through coordinated block delivery.
Data Source
AI summary
Described is a technology by which a scalable distributed file system is provided by deploying intelligent storage devices that are directly connected to the network. A server receives a request for data from a requester, and sends corresponding requests to the storage devices. The storage devices send data blocks directly to the requester instead of back to the server, while identifying the server as the sender, e.g., by programming a switch to participate in such an operation.