Hierarchical Database Querying With Parallel Asynchronous IO
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data storage mechanisms face inefficiencies in data query performance due to sequential processing and lack of parallelization in key-value database systems, leading to delays and suboptimal performance.
Innovation Solution
Implementing a parallel asynchronous IO query for multiple hierarchical storage layers in a key-value database system, where data are stored as master and additional data blocks with metadata for location information, allowing simultaneous queries across layers to enhance data retrieval efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If sequential data query processing is used in conventional database systems, then system simplicity is maintained, but data query performance deteriorates due to delays
Solution Approach 1:
The patent segments the data storage structure into multiple hierarchical layers (memory layer, buffer layer, data storage medium layer) with each layer containing multiple data files. This segmentation enables parallel query processing across different layers simultaneously, improving data query performance by eliminating sequential processing bottlenecks while maintaining manageable system complexity through organized layering
Solution Approach 2:
The patent introduces a hierarchical dimension to the data storage structure, organizing data across multiple levels (L0, L1, L2, etc.) with increasing storage capacity and decreasing access speed. This dimensional organization allows queries to be processed in parallel across different hierarchical levels, transforming the single-threaded sequential query model into a multi-dimensional parallel query architecture that improves performance without proportionally increasing complexity
2Quantity of substance
If data are stored in a hierarchical structure across multiple layers, then data storage capacity is improved, but data query time increases due to layer-by-layer processing
Solution Approach 1:
The patent implements preliminary actions by pre-computing and storing metadata (including perfect hash values and common hash values) at each hierarchical layer before queries are executed. This preliminary organization of data with embedded indexing information enables direct location of data blocks without sequential layer-by-layer searching, significantly reducing query time while maintaining the hierarchical storage structure's high capacity
Solution Approach 2:
The patent introduces metadata as an intermediary layer between the query interface and the hierarchical data storage. This metadata contains pre-computed hash values and location information that mediates the query process by directly mapping query keys to specific data block locations across hierarchical layers, eliminating the need for time-consuming sequential searches through all storage layers
3Speed
If parallel asynchronous IO queries are implemented across hierarchical layers, then data query speed is improved, but system complexity increases
Solution Approach 1:
The patent implements self-service mechanisms where each hierarchical layer autonomously processes query requests independently using its own metadata and data blocks. The parallel asynchronous IO queries are designed to be self-contained operations that can execute simultaneously without complex inter-layer coordination, reducing the overall system complexity despite enabling parallel processing across multiple layers at high speed
Data Source
AI summary
A computer-implemented method for querying data, includes performing a data query in a memory table in response to receiving a data query request initiated by a user. At least one master data block asynchronous input/output (IO) query for a hierarchical storage layer is initiated in parallel to a database system in response to that a data query result for the memory table does not satisfy a data query condition in the data query request. After receiving a query response to all master data block asynchronous IO queries, data analysis is performed on a master data block obtained through a query. A query result is determined for the data query request based on the data query result for the memory table or a data analysis result of the master data block. The data query result for the data query request is provided to the user.