Combined Erasure-Coded Blocks for Selective Data Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage methods are inefficient in terms of input/output operations, memory usage, processing, network bandwidth, and the number of operations required for retrieving portions of a large data object, especially when using erasure coding and distribution across multiple storage locations.
Innovation Solution
The method involves generating erasure coded data portions, which are then split into parts and stored in different locations as combined blocks, allowing for efficient retrieval of specific parts without needing to access the entire block, using metadata to identify and retrieve only the necessary parts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored in distributed locations with erasure coding for backup purposes, then data reliability is improved, but the number of input/output operations and processing requirements increase when retrieving data portions
Solution Approach 1:
The patent divides the data object into multiple data portions, where each data portion is independently erasure coded and stored in different locations. This segmentation allows selective retrieval of only the required data portions rather than the entire data object, thereby improving retrieval efficiency while maintaining data reliability through distributed storage and erasure coding.
2Stability of the object's composition
If the entire data object is retrieved to access specific portions, then data integrity is maintained, but memory usage and processing requirements increase
Solution Approach 1:
The patent extracts only the specific data portions that are needed from the distributed storage locations, rather than retrieving the entire data object. Each data portion is independently erasure coded, allowing the system to extract and reconstruct only the required portions, thereby reducing memory usage and processing requirements while maintaining data integrity for the accessed portions.
3Reliability
If data is erasure coded and split into multiple parts stored in different locations, then data reliability is improved, but the number of operations required for retrieval increases
Solution Approach 1:
The patent segments the data object into multiple independently erasure-coded portions stored in different locations. This segmentation enables parallel retrieval of required portions from multiple locations simultaneously, reducing overall retrieval time compared to sequential access, while maintaining data reliability through the erasure coding of each segment.
Data Source
AI summary
Examples are disclosed including generation of a first erasure coded data portion based on a first data portion of the data object, generation of a second erasure coded data portion based on a second data portion of the data object, formation of a first combined block by combining a first part of the first erasure coded data portion and a first part of the second erasure coded data portion, formation of a second combined block by combining a second part of the first erasure coded data portion and a second part of the second erasure coded data portion, storage of the first combined block in a first storage location, and storage of the second combined block in a second storage location.