Distributed Data-Parity Storage Layout for Faster Database Writes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database systems face limitations in processing speed due to hardware constraints, data storage methods, and restricted co-processing options, which hinder efficient data processing and retrieval.
Innovation Solution
The implementation of a parallelized database system architecture that divides data into partitions and segments, using a 4 of 5 encoding scheme for redundancy, and distributes parity blocks across separate storage sections to enhance data access efficiency and reduce processing time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is stored in a single storage section, then storage simplicity is maintained, but data access speed and processing efficiency deteriorate
Solution Approach 1:
The storage system is divided into multiple independent storage sections, with each section storing specific data blocks or parity blocks. This segmentation enables parallel access to different data portions, significantly improving data access speed while maintaining manageable complexity through modular organization
Solution Approach 2:
The patent introduces a new dimension to storage architecture by adding redundancy through parity blocks stored in separate sections. This transforms the storage system from a single-dimension sequential access model to a multi-dimensional parallel access model, enhancing both speed and reliability
2Productivity
If parity blocks are stored with data blocks in the same section, then storage management is simplified, but write operation bottlenecks increase
Solution Approach 1:
Parity blocks are extracted from the data storage sections and placed in dedicated parity storage sections. This separation eliminates the write bottleneck caused by simultaneous updates to both data and parity in the same section, improving write operation throughput while distributing parity management across independent sections
Solution Approach 2:
Dedicated parity sections act as intermediary storage areas that handle redundancy management separately from data storage. This intermediary structure allows write operations to proceed more efficiently by decoupling data writing from parity updating, reducing contention and bottlenecks
3Productivity
If hardware resources are increased to improve processing speed, then data processing speed improves, but system cost and complexity increase
Solution Approach 1:
The system segments data into multiple blocks distributed across different storage sections, enabling parallel processing operations. This segmentation allows the database system to process multiple data blocks simultaneously, improving overall processing speed without requiring proportional increases in single-unit hardware capabilities
Solution Approach 2:
The patent changes the organizational parameters of data storage by implementing a distributed block structure with specific redundancy ratios (e.g., 4 data blocks + 1 parity block). This parameter-based approach optimizes processing speed through parallelization while maintaining controlled hardware requirements through efficient space utilization
Data Source
AI summary
A method includes generating, by a processing entity of a computing system, a plurality of parity blocks from a plurality of lines of data blocks. A first number of parity blocks of the plurality of parity blocks is generated from a first line of data blocks of the plurality of lines of data blocks. The method further includes storing, by the processing entity, the plurality of lines of data blocks in data sections of memory of a cluster of computing devices of the computing system in accordance with a read/write balancing pattern and a restricted file system. The method further includes storing, by the processing entity, the plurality of parity blocks in parity sections of memory of the cluster of computing devices in accordance with the read/write balancing pattern and the restricted file system.