CDN Data Segment Caching for Faster Dispersed Storage Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current dispersed storage networks face challenges in securely and reliably storing and retrieving large amounts of data across geographically diverse locations, particularly in maintaining data integrity and efficiency in distributed task processing.
Innovation Solution
A distributed computing system that employs dispersed error encoding and decoding, where data is segmented, encoded, and distributed across multiple storage units, allowing for secure storage and retrieval while maintaining data integrity through error correction and secure task execution across geographically dispersed units.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is distributed across multiple geographically diverse storage units, then system reliability and fault tolerance are improved, but data integrity and security become more difficult to maintain
Solution Approach 1:
The patent segments data into multiple slices and distributes them across different storage units. Each slice is independently stored, and the system can reconstruct the original data from any sufficient subset of slices. This segmentation approach allows the system to maintain high reliability through geographic distribution while preserving data integrity through controlled access and reconstruction mechanisms.
Solution Approach 2:
The patent introduces intermediary components including access controllers, authentication services, and data reconstruction mechanisms that mediate between distributed storage units and users. These intermediaries enforce security policies, verify data integrity, and coordinate reconstruction processes, thereby maintaining data integrity and security across the distributed system.
2Reliability
If data is encoded and distributed across multiple storage units, then fault tolerance is improved, but storage and retrieval complexity increases
Solution Approach 1:
The patent divides data into slices and further segments them into distributed units with unique identifiers. This multi-level segmentation simplifies the management of distributed data by creating modular, independently addressable units that can be stored, retrieved, and managed separately, reducing overall system complexity despite the distributed architecture.
Solution Approach 2:
The patent creates multiple copies of data slices distributed across different storage units. Each copy is independently manageable and can be retrieved without accessing other copies. This copying approach simplifies fault tolerance by allowing independent access to any sufficient subset of copies, reducing the complexity of coordinated retrieval operations.
3Loss of information
If error correction encoding is applied to distributed data, then data security is improved, but processing time and computational resources increase
Solution Approach 1:
The patent applies error correction encoding and data slicing in advance during the data ingestion phase. By performing these computationally intensive operations beforehand, the system minimizes processing time during data retrieval and reconstruction operations. The pre-encoded slices can be directly stored and retrieved without requiring real-time computational processing.
Solution Approach 2:
The patent segments data into smaller slices that can be independently encoded and processed. This segmentation reduces the computational burden on individual processing units and allows parallel processing of multiple slices, thereby reducing overall processing time while maintaining data security through distributed error correction.
4Reliability
If data is stored in dispersed locations, then system availability is improved, but access efficiency and retrieval speed decrease
Solution Approach 1:
The patent segments data into slices with unique identifiers and distributes them across multiple locations. This segmentation enables the system to retrieve data from the nearest or most available storage units without requiring access to all distributed locations, thereby maintaining high system availability while improving retrieval speed through selective access to relevant segments.
Data Source
AI summary
A method for execution by a computing device includes, receiving, from a requesting device, a request for a data segment of a data object that is or is to be stored in storage units of a content delivery network. The method further includes determining whether the data segment is stored in a cache memory of the content delivery network or in the storage units. When stored in the cache memory, the method includes retrieving the cached data segment, and sending it to the requesting device. When stored in the storage units, the method includes, sending read requests regarding the data segment to the storage units, receiving, in response to the read requests, at least a decode threshold number of encoded data slices, decoding the at least the decode threshold number of encoded data slices to reproduce the data segment, and sending the data segment to the requesting device.