HBase-based indexing system and query acceleration method

Abstract

Based on the HBase index system and query acceleration method, by utilizing the orderliness of the HBase row key, the index data column family is built in the HBase table to create an index column for a business data column under the business data column family, and the index Write the index column row key value under the row key column of the HBase table corresponding to the column, and the index column row key value includes the indexed service data column column value and the service data column located after the service data column column value Row key value, and construct the present invention with the HBase table of the built-in index data column family as the core, can reach the purpose of searching more quickly to the data resources in the HBase table under complex query conditions, thereby being conducive to more fully utilizing and developing HBase data resources.

Description

technical field [0001] The invention relates to HBase database technology, in particular to an HBase-based indexing system and a query acceleration method. Background technique [0002] Apache HBase is a highly reliable, high-performance, column-oriented, scalable, open source non-relational distributed database (referred to as HBase database or HBase), which is mainly used to store unstructured and semi-structured loose data. In a horizontally scalable way, cheap computer clusters are used to process data tables consisting of more than 1 billion rows of data and millions of column elements. HBase itself provides queries based on full table scans, as well as queries based on row keys (similar to the primary keys of tables in relational databases). However, in practical application scenarios, data query conditions are often more complex. For the processing of complex query conditions, in HBase, data can only be retrieved through full table scanning, that is, all data in the ...

AI Technical Summary

Problems solved by technology

The inventor realizes that although the above-mentioned HBase secondary indexes are the original full-table scanning mode of the HBase table, the improvement of a large number of disk I / O operations and the long waiting time of the query is brought, but there are some defects. Or insufficient, for example, the HBase secondary index based on the MapReduce batch processing method cannot build the index in real time, and the effectiveness is not high; for example, the HBase secondary index pre-created by the third-party Solr engine needs to rely on the third-party engine, the coupling is too strong, Too many resource requirements; for example, using tables (HBase tables, such as user tables, etc.) in the same HBase database to build external tables (secondary index tables for user tables) may result in physical separation of data tables and index tables, while The separation of index tables and data tables may lead to data consistency problems, etc., and is not conducive to index reverse lookup

The order of HBase row keys is the basis for HBase to provide query based on row keys. However, the HBase row keys corresponding to HBase tables in the prior art do not include business data, that is, they do not include the column value of a column under the column family in the table. (business data), while the data in columns other than the row key in HBase is unordered, that is to say, the existing HBase table itself cannot provide any index retrieval for business data columns

Images