Hybrid Database Storage for Query Speed and Space Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems face limitations in processing speed due to hardware constraints, data storage methods, and restricted co-processing options, leading to inefficiencies in query execution.
Innovation Solution
A database system utilizing replication-based and redundancy coding-based storage systems, combined with parallelized data input, storage, retrieval, and query processing, to optimize data partitioning, storage, and query execution through efficient resource allocation and optimized query plans.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If replication-based storage is used, then data availability and query processing speed are improved, but storage space efficiency deteriorates
Solution Approach 1:
The patent segments data into multiple partitions that can be stored across different storage systems. Some partitions use replication-based storage for high-speed query access, while others use redundancy coding-based storage for space-efficient archival. This segmentation allows the system to optimize different data portions for different purposes, resolving the contradiction between speed and storage efficiency.
Solution Approach 2:
The patent applies different storage quality characteristics to different data partitions based on their access patterns and importance. Frequently accessed data is stored with replication for high availability and speed, while less frequently accessed data uses redundancy coding for space efficiency. This local differentiation of storage quality resolves the contradiction by applying the appropriate storage method to each data portion.
2Quantity of substance
If redundancy coding-based storage is used, then storage space efficiency is improved, but data retrieval speed deteriorates
Solution Approach 1:
The patent divides data into partitions that are strategically placed in different storage systems. Hot data that requires fast retrieval is segmented and placed in replication-based storage, while cold data is segmented and placed in redundancy coding-based storage. This segmentation ensures that data retrieval speed requirements are met for appropriate data portions while maximizing storage efficiency elsewhere.
Solution Approach 2:
The patent changes the storage parameter (replication factor vs. redundancy coding ratio) based on data characteristics and access patterns. By dynamically adjusting these parameters for different data partitions, the system optimizes the balance between retrieval speed and storage efficiency, resolving the contradiction for overall system performance.
3Productivity
If parallelized data input and processing is implemented, then query processing efficiency is improved, but system complexity deteriorates
Solution Approach 1:
The patent segments the data input and processing workflow into parallel streams that can be independently processed. Multiple data input channels, processing nodes, and query execution paths operate in parallel, each handling specific data partitions. This segmentation enables parallelized processing that improves productivity while managing complexity through modular design.
Solution Approach 2:
The patent implements universal processing components that can handle multiple types of operations across different data partitions. The same processing architecture serves both replication-based and redundancy coding-based storage systems, reducing overall system complexity while enabling parallelized processing for improved productivity.
Data Source
AI summary
A database system operates by: receiving a set of records for storage; applying a replication factor dictated by the required fault tolerance level to generate a set of replicas of the set of records; temporarily storing the set of replicas in a temporary ingress storage system during a first temporal period in accordance with the required fault tolerance level; generating a plurality of redundancy-coded segments that encodes the set of record utilizing a redundancy storage coding scheme dictated by the required fault tolerance level; and migrating the set of records from the temporary ingress storage system to a long-term storage system by storing the set of segments in the long-term storage system and by removing the set of replicas from the temporary ingress storage system.