Distributed File Access Nodes for Parallel Download Performance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing file storage services limit download performance by relying on a single connection and lack efficient distribution of file access across multiple nodes, leading to unreliable and inefficient file access operations.
Innovation Solution
Implementing a distributed system with multiple file access nodes that can independently serve file portions to clients from local caches or act as proxies, optimizing bandwidth usage and reliability by distributing file access requests and monitoring performance metrics to manage workload.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a single connection is used for file access, then the system complexity is low, but the download performance and bandwidth usage are limited
Solution Approach 1:
The patent segments the file access operation by dividing the file into multiple portions and distributing these portions across multiple file access nodes. Each node independently serves specific file portions to the client, enabling parallel downloads and improving overall download performance while managing system complexity through modular node architecture
Solution Approach 2:
The patent transitions from a single-connection model to a multi-node distributed architecture, adding spatial dimensionality to the file access system. Multiple file access nodes operate in parallel across different network paths, transforming the bottlenecked single-connection approach into a multi-dimensional bandwidth aggregation system
2Reliability
If multiple file access nodes are deployed, then the reliability and bandwidth usage are improved, but the system complexity increases
Solution Approach 1:
The system segments file access functionality into independent file access nodes, each capable of serving file portions autonomously. This segmentation improves reliability through redundancy - if one node fails, other nodes continue serving their assigned portions - while keeping each node's internal complexity manageable through standardized node architecture
Solution Approach 2:
The control node acts as an intermediary that manages the distributed file access nodes. It receives client requests, determines optimal node assignments, and coordinates file portion distribution. This intermediary approach simplifies the overall system complexity by centralizing coordination logic while allowing individual nodes to remain relatively simple and reliable
3Speed
If file portions are served from local caches, then the bandwidth usage is optimized and access speed is improved, but the cache management complexity increases
Solution Approach 1:
File portions are pre-cached in the local storage of file access nodes before client requests arrive. This preliminary action enables immediate serving of cached portions without real-time network retrieval, significantly improving file access speed. The cache management complexity is reduced by allowing nodes to autonomously maintain their local caches without complex centralized cache coordination
Data Source
AI summary
Systems and methods for facilitating access to files that are stored by network-accessible file storage services. An example system comprises a plurality of file access nodes including a control node, wherein the control node is configured to: receive a file access request initiated by a client, wherein the file access request comprises an identifier of a file, an identifier of a file storage service, and an access token associated with the file stored by the file storage service; request, using the access token, metadata associated with the file from the file storage service; define, using the metadata, a plurality of portions forming the file; and forward, to the plurality of file access nodes, identifiers of the plurality of portions.