Distributed Data Storage With Variable Redundancy and Synchronized Offsets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for storing and rebuilding video surveillance data, such as RAID and backup schemes, face inefficiencies in disk space usage and redundancy, particularly when handling data loss or corruption, and lack flexibility in adjusting redundancy levels based on data importance.
Innovation Solution
A computer-implemented method that splits data into N pieces and generates M redundancy pieces, storing them on separate storage media with synchronized offsets, allowing for flexible redundancy adjustment and efficient use of disk space, using erasure coding algorithms like Reed Solomon to rebuild lost data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If full duplicates of the data are stored (RAID1), then data reliability is improved, but disk capacity requirement doubles
Solution Approach 1:
The patent divides data into multiple blocks and distributes them across different storage media, rather than creating full duplicates. This segmentation allows for more efficient use of storage space while maintaining reliability through distributed redundancy.
Solution Approach 2:
The patent changes the redundancy parameter from fixed (RAID1 requires 100% duplicate, RAID5/6 have fixed parity) to variable, allowing the system to adjust the number of redundant blocks based on data importance and available storage capacity, thus optimizing the balance between reliability and storage efficiency.
2Quantity of substance
If RAID 5 and 6 are used, then storage overhead is reduced, but flexibility in redundancy adjustment is lost
Solution Approach 1:
The patent implements a dynamic redundancy system where the number of redundant blocks can be adjusted based on data importance and storage availability. Unlike static RAID configurations, this system can adaptively change redundancy levels without requiring complete system reconfiguration or rebuilds.
Solution Approach 2:
The system allows dynamic modification of redundancy parameters, enabling users to increase or decrease the number of redundant blocks based on changing requirements. This parameter flexibility resolves the contradiction between low storage overhead and adaptability.
3Adaptability or versatility
If erasure coding is used, then arbitrary combinations of data and redundant blocks are possible, but rebuilding complexity increases
Solution Approach 1:
The patent segments data into manageable blocks with associated metadata that tracks their relationships. This segmentation simplifies the rebuilding process by allowing the system to identify and reconstruct only the specific missing blocks needed, rather than complex global recalculations.
Solution Approach 2:
The system performs preliminary organization of data blocks and metadata during the initial storage phase, establishing clear relationships between data and redundant blocks. This preliminary structuring simplifies subsequent rebuilding operations by providing a ready framework for reconstruction.
4Device complexity
If fixed size disk sets are used, then system simplicity is maintained, but ability to handle drive failures gracefully is reduced
Solution Approach 1:
The patent implements dynamic disk set configuration that can adapt when drives are added or removed. The system can recalculate and redistribute data and redundancy blocks across the available drives, maintaining reliability without requiring fixed-size configurations.
Solution Approach 2:
The system design allows storage media to serve multiple functions - data blocks, redundant blocks, or both - depending on configuration needs. This universality enables flexible handling of drive failures while maintaining system simplicity.
Data Source
AI summary
A computer-implemented method of storing an item of data across a plurality of storage media, the method comprising the steps of: receiving an item of data to be stored; splitting the item of data into N pieces of data; generating M redundancy pieces of data, usable to rebuild the item of data; storing each of the N pieces of data and M redundancy pieces of data on separate storage media of the plurality of storage media at a same offset within a file of the respective storage medium, the files of the storage media containing the N pieces of data and the M redundancy pieces of data being associated as a file set; and storing, separately to the N pieces of data and M redundancy pieces of data, the offset and the file set.