Proxy Script File Retrieval from Distributed Storage Nodes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data retrieval methods face challenges in balancing reliability and read performance, as data striping provides high performance but is vulnerable to storage device failures, while duplicating data across multiple devices does not offer improved read performance.
Innovation Solution
A method involving a client computer sending requests to proxy nodes, which generate scripts for retrieving file portions from multiple storage nodes, allowing concurrent data retrieval and concatenation to reconstruct the file, enabling efficient retrieval without relying on a single drive.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data striping is used to distribute data segments across multiple storage devices, then read performance is improved through concurrent fetching, but reliability deteriorates due to vulnerability to storage device failures
Solution Approach 1:
The patent divides the file into multiple segments and distributes them across different storage devices using data striping. Each segment can be fetched concurrently from different devices, improving read performance while maintaining the ability to reconstruct the file even if some devices fail
Solution Approach 2:
The system dynamically adjusts the number of segments and their distribution across storage devices based on performance requirements and failure tolerance parameters, allowing optimization of both read performance and reliability for different scenarios
2Reliability
If multiple copies of data are stored on different computers and drives, then reliability is improved through redundancy, but read performance deteriorates since data must be read from a single drive
Solution Approach 1:
Instead of storing complete copies of the file on each drive, the patent segments the file and distributes segments across multiple drives. The client can then retrieve different segments from different drives simultaneously, achieving both reliability through distribution and high read performance through parallel access
Solution Approach 2:
The patent transitions from a single-dimension approach (reading entire file from one drive) to a multi-dimensional approach where different segments of the file are read from different drives in parallel, effectively adding the dimension of parallel data retrieval
3Productivity
If data striping is implemented using RAID or parallel file systems, then read performance is improved, but device complexity and cost increase
Solution Approach 1:
The patent introduces a file segmentation and distribution layer that acts as an intermediary between the client and storage devices. This layer handles the complexity of segment management and distribution, allowing standard storage devices to be used without requiring complex RAID hardware or parallel file system software
Solution Approach 2:
The system allows the client or a coordination service to automatically manage the segmentation, distribution, and retrieval of file segments across multiple storage devices, eliminating the need for complex centralized management systems while still achieving high read performance
Data Source
AI summary
A system and associated computer program product for retrieving a file from a distribution of storage nodes. A client computer sends a request for a file to a proxy layer and in response, the proxy layer generates a script for obtaining the requested file and sends the script to the client computer. The client computer executes the script, resulting in portions of the requested file being obtained from respective storage nodes in a storage layer. The portions of the requested file are concatenated to form the requested file.