Hybrid Row-Column Querying with Row Index for Point Reads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Columnar storage systems exhibit low data query efficiency in scenarios where complete rows of data need to be queried, particularly in high-concurrency point query scenarios.
Innovation Solution
The method involves storing data in both row and columnar storage forms within a columnar storage system, constructing a row storage index, and matching data query requests with this index to improve query efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is stored in columnar storage form, then data reading efficiency is improved when only some columns need to be read, but data query efficiency deteriorates when complete rows need to be queried
Solution Approach 1:
The patent segments the storage system into two distinct storage forms: columnar storage for efficient columnar queries and row storage for efficient row-based queries. By dividing the storage architecture, the system can handle different query types optimally without compromising performance in either scenario.
Solution Approach 2:
The patent creates a multi-functional storage system where data exists in both columnar and row storage forms simultaneously. This universal approach allows the same data to be efficiently queried regardless of whether the query requires complete rows or specific columns, making the storage system adaptable to different query patterns.
2Productivity
If data is stored in both row storage form and columnar storage form, then data query efficiency is improved, but storage resource usage increases
Solution Approach 1:
The patent applies preliminary action by pre-storing data in both row and columnar formats before queries are executed. This advance preparation ensures that regardless of the query type, the data is already in the optimal format for retrieval, eliminating the need for costly data transformation during query execution.
Solution Approach 2:
The patent changes the storage parameter by maintaining data in two different organizational forms (row-based and column-based). This parameter change allows the system to optimize for different query patterns without requiring data duplication across separate systems, thereby managing storage resources more efficiently.
3Productivity
If row storage index is constructed and maintained, then complete row data query efficiency is improved, but device complexity increases
Solution Approach 1:
The patent introduces a row storage index as an intermediary structure that bridges the gap between row storage and query operations. This index acts as a mediator that speeds up row-based queries without requiring fundamental changes to the underlying storage architecture, thereby managing complexity more effectively.
Solution Approach 2:
The patent creates a simplified copy or representation of row data through the row storage index. This index copy contains only the essential information needed for quick row lookups, allowing efficient query execution without maintaining full complexity of the underlying row storage structure in the index itself.
Data Source
AI summary
The present application discloses a data query method. The method can be applied to a columnar storage system. The columnar storage system stores target data that is stored in a row storage form, target data that is stored in a columnar storage form, and a row storage index constructed based on the target data that is stored in a row storage form. The columnar storage system can receive a data query request and matches the data query request with the row storage index corresponding to the columnar storage system. If the matching is successful, data matched with the data query request is queried based on the row storage index. Therefore, by use of the scheme, the target data can be stored in the row storage form into the columnar storage system, and the row storage index can be constructed.