Distributed Data-Parity Storage for Lock-Free Parallel Databases
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database systems face limitations in processing speed due to hardware constraints, data storage methods, and restricted co-processing options, which hinder efficient data processing and retrieval.
Innovation Solution
A parallelized database system architecture that divides data into partitions, uses a 4 of 5 encoding scheme to create data and parity segments, and distributes these segments across multiple computing devices for parallel processing and storage, optimizing data storage and retrieval through separate data and parity storage sections.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is stored in a traditional centralized database system, then data storage is simple, but processing speed is limited by hardware constraints and sequential operations
Solution Approach 1:
The database system is divided into multiple computing devices, each handling specific partitions of data. Administrative operations are segmented into lock-free operations that can execute in parallel across devices, eliminating sequential bottlenecks and improving processing speed while distributing system complexity across multiple independent units.
Solution Approach 2:
The system transitions from a single-dimensional centralized storage model to a multi-dimensional distributed architecture where data is organized across computing devices, partitions, segments, and sections. This dimensional expansion enables parallel processing paths and eliminates sequential operation constraints.
2Reliability
If administrative operations use traditional locking mechanisms, then data consistency is maintained, but processing time increases due to sequential execution
Solution Approach 1:
The locking mechanism is extracted and replaced with a lock-free operational model. Administrative operations are designed to execute without acquiring locks, allowing multiple operations to proceed concurrently across computing devices while maintaining data consistency through alternative mechanisms such as version control or conflict-free replicated data types.
Solution Approach 2:
Administrative operations continue executing continuously without being blocked by lock acquisition and release cycles. The lock-free design enables uninterrupted parallel execution of operations across multiple computing devices, eliminating idle time and maintaining continuous productive action.
3Productivity
If data is stored without separation of data and parity sections, then storage structure is simple, but data access efficiency and redundancy handling are reduced
Solution Approach 1:
Storage is segmented into distinct data sections and parity sections within each computing device. This segmentation allows independent access to data and parity information, enabling efficient data retrieval operations without requiring parity calculations and allowing separate optimization of data access paths versus error handling paths.
Solution Approach 2:
Different sections of storage are assigned different qualities and purposes: data sections are optimized for rapid read access and write operations, while parity sections are optimized for error detection and recovery. This local quality differentiation improves overall data access efficiency by allowing operations to access only the necessary section.
Data Source
AI summary
A method includes generating a plurality of parity blocks from a plurality of lines of data blocks. The plurality of lines of data blocks are stored in data sections of memory of a cluster of computing devices of the computing system by distributing storage of individual data blocks of the plurality of lines of data blocks among unique data sections of the cluster of computing devices. The plurality of parity blocks are stored in parity sections of memory of the cluster of computing devices by distributing storage of parity blocks of the plurality of parity blocks among unique parity sections of the cluster of computing devices.