Index system based on HBase and query acceleration method

Abstract

The invention discloses an index system based on HBase and a query acceleration method. According to the method, the orderliness of HBase row keys is utilized; an index data column family is built inan HBase table so as to establish an index column for a service data column in a service data column family; an index column row key value is written into a row key column of the HBase table corresponding to the index column, wherein the index column row key value comprises an indexed business data column value and a business data column row key value positioned behind the business data column value; and the HBase table with the built-in index data column family is used as a core to construct the method, so that the aim of more quickly searching the data resources in the HBase table under complex query conditions can be fulfilled, and the data resources of the HBase can be more fully utilized and developed.

Description

technical field [0001] The invention relates to HBase database technology, in particular to an HBase-based indexing system and a query acceleration method. Background technique [0002] Apache HBase is a highly reliable, high-performance, column-oriented, scalable, open source non-relational distributed database (referred to as HBase database or HBase), which is mainly used to store unstructured and semi-structured loose data. In a horizontally scalable way, cheap computer clusters are used to process data tables consisting of more than 1 billion rows of data and millions of column elements. HBase itself provides queries based on full table scans, as well as queries based on row keys (similar to the primary keys of tables in relational databases). However, in practical application scenarios, data query conditions are often more complex. For the processing of complex query conditions, in HBase, data can only be retrieved through full table scanning, that is, all data in the ...

AI Technical Summary

Problems solved by technology

The inventor realizes that although the above-mentioned HBase secondary indexes are the original full-table scanning mode of the HBase table, the improvement of a large number of disk I / O operations and the long waiting time of the query is brought, but there are some defects. Or insufficient, for example, the HBase secondary index based on the MapReduce batch processing method cannot build the index in real time, and the effectiveness is not high; for example, the HBase secondary index pre-created by the third-party Solr engine needs to rely on the third-party engine, the coupling is too strong, Too many resource requirements; for example, using tables (HBase tables, such as user tables, etc.) in the same HBase database to build external tables (secondary index tables for user tables) may result in physical separation of data tables and index tables, while The separation of index tables and data tables may lead to data consistency problems, etc., and is not conducive to index reverse lookup

The order of HBase row keys is the basis for HBase to provide query based on row keys. However, the HBase row keys corresponding to HBase tables in the prior art do not include business data, that is, they do not include the column value of a column under the column family in the table. (business data), while the data in columns other than the row key in HBase is unordered, that is to say, the existing HBase table itself cannot provide any index retrieval for business data columns

Images