Distributed Segment Group Storage With Parity for Faster Queries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database systems face limitations in processing speed due to hardware constraints, data storage methods, and restricted co-processing options, which hinder efficient data processing and retrieval.
Innovation Solution
The implementation of a parallelized database system architecture that divides data into segments, uses a 4 of 5 encoding scheme, and distributes data and parity blocks across multiple computing devices, allowing for efficient storage and retrieval through a distributed parity storage section and data storage section, optimizing data access and processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is stored in a centralized database system, then data storage capacity is sufficient, but processing speed is limited due to hardware constraints and sequential operations
Solution Approach 1:
The patent divides the database system into multiple storage clusters, each containing multiple computing devices. Data is segmented into different segment groups that are distributed across these clusters. This segmentation enables parallel processing of different data segments simultaneously, thereby improving processing speed while managing system complexity through modular organization.
Solution Approach 2:
The patent introduces a distributed architecture dimension by organizing storage systems into multiple clusters with multiple computing devices each. This multi-dimensional structure allows concurrent operations across different clusters and devices, transforming the single-point processing bottleneck into a parallel processing network that improves throughput.
2Productivity
If data is divided into segments and stored across multiple computing devices, then processing parallelism is improved, but data retrieval complexity increases
Solution Approach 1:
The patent implements a coordinator component that tracks the location of different segment groups across storage clusters. When data retrieval is requested, the coordinator provides feedback information about where specific segments are stored, enabling efficient direct access without requiring complex distributed queries. This feedback mechanism simplifies the retrieval process while maintaining parallel processing benefits.
3Reliability
If parity blocks are stored separately from data blocks, then data redundancy and fault tolerance are improved, but storage access efficiency decreases
Solution Approach 1:
The patent segments parity storage into separate parity sections within each storage cluster, distinct from data sections. This segmentation allows independent management and access patterns for data and parity information. When data is accessed, only the necessary data segments need to be retrieved, while parity segments remain accessible but are not blocked, maintaining both reliability and access efficiency.
Data Source
AI summary
A computing system is operable to generate a first plurality of lines of coding blocks that includes a first plurality of data blocks and a first plurality of parity blocks. A first set of segments of a first segment group that collectively include the first plurality of lines of coding blocks are generated, and the first set of segments of the first segment group are stored via a first plurality of computing devices of a first storage cluster of the computing system. A second plurality of lines of coding blocks that includes a second plurality of data blocks and a second plurality of parity blocks are generated. A second set of segments of a second segment group that collectively include the second plurality of lines of coding blocks are generated. The second set of segments of the second segment group are stored via a second plurality of computing devices of a second storage cluster of the computing system.