Segmented Cloud Storage With Parity for Bandwidth Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud storage providers face high costs due to bandwidth and hardware requirements for storing and accessing large files, as data is typically centralized, leading to bottlenecks and increased risk of data unavailability from single storage locations.
Innovation Solution
A system that distributes digital files into multiple distinct blocks with parity information across remote storage nodes, allowing for efficient bandwidth usage and error correction, using techniques like Reed-Solomon encoding and decoding, and dynamically selecting available nodes for data retrieval.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If data is centralized in cloud storage, then data security and management are simplified, but bandwidth costs and hardware requirements increase significantly
Solution Approach 1:
The patent divides files into multiple segments and distributes them across different storage nodes in a peer-to-peer network. This segmentation eliminates the centralized bottleneck, reducing bandwidth costs while maintaining data security through distributed architecture where no single node holds the complete dataset.
Solution Approach 2:
The system transitions from a single-dimension centralized storage model to a multi-dimensional distributed network architecture. Data is organized across multiple nodes with hierarchical indexing, enabling efficient retrieval without centralized coordination while reducing bandwidth requirements through parallel access paths.
2Ease of operation
If data is stored in a single location, then data access is simplified, but data availability and security are reduced due to single point of failure risk
Solution Approach 1:
Files are divided into segments distributed across multiple nodes, with each node storing only portions of the overall dataset. This segmentation provides redundancy and fault tolerance while maintaining simplified access through the hierarchical index system that guides retrieval operations across the distributed network.
Solution Approach 2:
The system implements redundant storage of data segments across multiple nodes before any failure can occur. This prior cushioning ensures data availability continues even if some nodes become unavailable, while the hierarchical index structure maintains operational simplicity for data retrieval.
3Reliability
If cloud providers purchase redundant storage arrays and large bandwidth, then data security and access speed are improved, but operational costs increase significantly
Solution Approach 1:
The system enables peer-to-peer data sharing where network participants contribute their own storage capacity and bandwidth resources. This self-service model eliminates the need for centralized providers to purchase expensive redundant infrastructure, reducing operational costs while maintaining data security through distributed architecture and cryptographic protection.
Solution Approach 2:
The hierarchical index system serves multiple functions simultaneously: it enables efficient data retrieval, provides distributed coordination without centralized control, and optimizes bandwidth utilization across the network. This multi-functionality reduces operational complexity and costs while maintaining high reliability and security standards.
Data Source
AI summary
A computer system for optimizing bandwidth usage within a cloud storage system receives a request, through an application program interface (API), to store a digital file. The computer system then encodes the digital file into a set of multiple distinct blocks of data. The system also transmits the set of multiple distinct blocks of data to multiple remote storage nodes. The multiple distinct blocks of data are divided among at least a portion of the remote storage nodes. The system generates a log that comprises identification information associated with the digital file and an address for each respective block of data within each respective remote storage node. The system then transmits the log to a remote directory server. The directory server comprises multiple different logs that map multiple different digital files to the remote storage nodes.