Distributed Data Segment Storage for Faster Database Queries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database systems face limitations in processing speed due to hardware constraints, data storage methods, and restricted co-processing options, which hinder efficient data processing and query execution.
Innovation Solution
A parallelized database system architecture that divides data into partitions, uses a distributed encoding scheme, and employs global dictionary compression (GDC) and run length encoding to optimize data storage and retrieval, allowing for parallel processing across multiple nodes and improved query execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is stored in traditional database systems with conventional storage methods, then data can be stored and retrieved, but processing speed is limited due to hardware constraints and sequential processing
Solution Approach 1:
The patent divides data into partitions and further segments each partition into multiple data segments. This segmentation enables parallel processing across multiple computing devices, thereby improving processing speed while managing system complexity through structured organization.
Solution Approach 2:
The patent introduces a distributed storage architecture that adds spatial distribution as a new dimension to data storage. By storing data segments across multiple computing devices in a cluster, the system achieves parallel processing capabilities and improved speed without proportionally increasing complexity.
2Productivity
If data is stored without compression, then data retrieval is straightforward, but storage space is wasted due to redundancy and processing efficiency is reduced
Solution Approach 1:
The patent applies compression algorithms that change the parameter of data representation by encoding data in compressed formats. This reduces storage requirements and improves processing efficiency by minimizing the amount of data that needs to be transferred and processed, while the compression overhead is managed through systematic implementation.
Solution Approach 2:
The patent performs compression encoding as a preliminary action during data storage. By compressing data before storing it in segments across the distributed system, the patent reduces the overall data volume that needs to be managed, thereby improving productivity without proportionally increasing complexity.
3Loss of time
If data is distributed across multiple nodes for parallel processing, then query execution speed improves, but data storage and retrieval complexity increases
Solution Approach 1:
The patent segments data into manageable units that can be independently processed across multiple nodes. This segmentation enables parallel query execution, reducing query execution time while maintaining manageable complexity through consistent segment structures and metadata management.
Solution Approach 2:
The patent implements a query optimization process that uses feedback from cost analysis to determine the most efficient execution plan. By analyzing query patterns and system state, the optimizer selects strategies that minimize execution time while avoiding unnecessary complexity in distributed query processing.
4Productivity
If conventional encoding schemes are used for data storage, then data can be stored reliably, but storage efficiency is low and processing speed is limited
Solution Approach 1:
The patent transforms data from uncompressed to compressed format by changing the representation parameters through encoding algorithms. This parameter change achieves both compression (reducing data volume) and improved processing throughput, while the encoding/decoding processes preserve data integrity without information loss.
Data Source
AI summary
A method includes a host computing device receiving a segment group of data. The method further includes the host computing device evaluating availability status of other computing devices in the storage cluster of computing devices. When one of the other computing devices is unavailable, the method further includes the host computing device dividing the segment group of data into a plurality of lines of data blocks. For a line of the data blocks, the method further includes the host computing device generating at least one parity block. The method further includes the host computing device sending a first data segment that includes first positioned data blocks to a first available computing device. The method further includes the host computing device sending a second data segment that includes second positioned data blocks to a second available computing device. The method further includes the host computing device storing a parity segment.