Distributed File Storage via Authorization Tokens
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage methods using external memory services are limited by security, cost, availability, and efficiency, often requiring users to register with multiple services, which can be cumbersome and inefficient, especially for high-availability files.
Innovation Solution
A method that automatically generates a distribution schedule for storing file fragments across multiple memory services using error correction methods, ensuring data availability and security by separating access management and data storage, with the file management server providing authorization tokens for direct storage and reconstruction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If file fragments are stored in multiple memory services using error correction methods, then data availability and security are improved, but system complexity increases due to distribution schedule generation and authorization token management
Solution Approach 1:
The file is divided into multiple fragments using error correction methods (e.g., Reed-Solomon coding), where each fragment is stored in a different memory service. This segmentation enables data reconstruction from any subset of fragments, improving availability without requiring centralized management of the entire file.
Solution Approach 2:
The file management server acts as an intermediary that generates distribution schedules and authorization tokens, coordinating between the user computer system and multiple memory services. This mediator handles the complexity of distributed storage while presenting a simplified interface to users.
2Productivity
If file fragments are transferred in parallel to multiple memory services, then storage time is reduced, but network bandwidth consumption increases
Solution Approach 1:
The file is segmented into multiple fragments that can be transferred in parallel to different memory services simultaneously. This parallel transfer approach充分利用 network bandwidth across multiple connections, achieving faster overall storage speed while distributing network load.
Solution Approach 2:
The system transfers more data than the minimum required (including error correction bits and redundant fragments) to enable parallel processing and faster reconstruction. This excessive action in terms of data volume achieves time efficiency and improved storage productivity.
3Reliability
If the file management server requests authorization tokens from each memory service, then access security is improved, but the number of authentication operations increases
Solution Approach 1:
The file management server pre-requests authorization tokens from all target memory services before the actual file fragment transfer begins. This preliminary authentication ensures that security credentials are validated in advance, improving access security while enabling smooth subsequent data transfer operations.
4Reliability
If metadata are protected and stored only in the user computer system and/or file management server, then data security against unauthorized access by memory services is improved, but access management complexity increases
Solution Approach 1:
The metadata (including error correction information and reconstruction instructions) are extracted from the file fragments and protected separately in the user computer system and/or file management server. This extraction ensures that even if memory services gain unauthorized access to fragments, they cannot reconstruct the original file without the protected metadata.
Data Source
AI summary
A file management server may include a processor, a network interface for operatively coupling the file management server to a user computer system and to memory services via a network. The file management server includes a file management application configured to receive an authorisation enquiry of the user computer system to store file fragments of a file via the network in a plurality of the memory services; and in response to the receipt of the authorisation enquiry, request an authorisation token from each of the memory services and forward, to the user computer system, authorisation tokens formed as URLs and obtained in response to the request. Each URL enables direct write or direct read access to a storage space of one of the memory services identified by the URL. Metadata that allows reconstruction of the file from the stored file fragments is protected against access by the memory services.