Segmented Data and Parity Layout for Parallel Database Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database systems face limitations in processing speed due to hardware constraints, data storage methods, and restricted co-processing options, which hinder efficient data processing and retrieval.
Innovation Solution
The implementation of a parallelized database system architecture that divides data into partitions, uses a 4 of 5 encoding scheme, and distributes data and parity blocks across multiple computing devices, allowing for efficient storage and retrieval through a separate parity storage section, thereby optimizing data access and processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is stored in a traditional centralized database system, then data storage is simple, but processing speed is limited by hardware constraints and sequential access patterns
Solution Approach 1:
The patent segments data into multiple partitions distributed across different computing devices. Each partition is further divided into data blocks and parity blocks that are stored across multiple nodes. This segmentation enables parallel processing of multiple data partitions simultaneously, thereby increasing processing speed while managing complexity through modular architecture design
Solution Approach 2:
The patent introduces a new dimension to data storage by implementing a distributed architecture across multiple computing devices rather than using a single centralized system. This dimensional expansion from single-node to multi-node storage enables parallel query processing and improves throughput without being constrained by single-hardware limitations
2Productivity
If data is divided into partitions and distributed across multiple computing devices, then processing throughput increases, but data access complexity increases
Solution Approach 1:
The patent implements a universal data access interface that works across all distributed partitions. The query processing mechanism can uniformly access any partition regardless of its location, and the system provides consistent data retrieval operations across the distributed architecture. This multi-functionality allows the system to handle diverse query types while maintaining a unified access model, thereby improving productivity without proportionally increasing access complexity
3Reliability
If a 4 of 5 encoding scheme is used with separate parity storage section, then data redundancy and fault tolerance improve, but storage space requirements increase
Solution Approach 1:
The patent segments data into 4 data blocks and generates 1 parity block, creating 5 total segments. These segments are distributed across multiple computing devices. The segmentation allows the system to achieve fault tolerance (can recover from loss of up to 1 segment) while efficiently utilizing storage space across the distributed network, balancing reliability requirements with storage capacity
4Speed
If administrative and configuration operations are executed in parallel without locking, then query processing speed increases, but data consistency management becomes more difficult
Solution Approach 1:
The patent implements a lock-free architecture where each computing device independently processes queries on its local partitions without requiring locks or coordination with other nodes. The distributed query processor autonomously retrieves and processes data from relevant partitions in parallel. This self-service approach enables high-speed parallel query processing while minimizing consistency management complexity through decentralized, independent operation
Data Source
AI summary
A computing system is operable to generate a plurality of lines of coding blocks that includes a plurality of data blocks and a plurality of parity blocks. Each of the plurality of lines of coding blocks includes a corresponding subset of data blocks a corresponding subset of parity blocks. A set of segments of a segment group are generated to collectively include the plurality of lines of coding blocks. Different coding blocks of each of the plurality of lines of coding blocks are included within different ones of the set of segments, and the plurality of parity blocks are dispersed across all of the set of segments. The set of segments are stored via a plurality of nodes sets, where different segments of the set of segments are stored via memory resources of different node sets of the plurality of node sets.