Method for constructing Hbase secondary index table

Abstract

The invention discloses a method for constructing an Hbase secondary index table, and relates to the field of databases. Hbase only provides query based on row keys and full-table scanning, full-tablescanning is needed when non-row keys are queried, and the query efficiency is low. The method comprises the following steps: establishing an index table for an original table on an hbase table; for each record of the original table, recording; generating corresponding index records on the index table, row keys of the index records comprise partition keys, index values, connectors and original table main keys, wherein the partition keys are start key fields of regions where the index records are located, the partition keys serve as the index table main keys, and pre-partitioning the partitionkeys in a random hash mode in combination with the index table; pre-defining the partition value; determining an index partition value in advance when an index table is established; and pre-partitioning the index table according to the index partition value when the index table is established. Parallel performance can be effectively improved, and query efficiency is improved.

Description

technical field [0001] The invention relates to the field of databases, in particular to a method for constructing an Hbase secondary index table. Background technique [0002] At present, the industry has proposed various solutions for the secondary index of Hbase. When the index table has only one partition when it is created, a large number of accesses will be concentrated on a single region. In addition, in the scenario of writing comparison frequency, as the data grows faster and faster, the number of splits will also increase, and splits are time-consuming. Resource consuming. If the pre-partition is pre-built from the beginning, but because the prefix of the primary key of the index table is often the same consecutively and the order increases, this will still cause a large number of reads, writes or other operations to access one or a few nodes in the cluster. A single machine where a hotspot region is located exceeds its own capacity, causing performance degradati...

AI Technical Summary

Problems solved by technology

When the index table has only one partition when it is created, a large number of accesses will be concentrated on a single region. In addition, in the scenario of writing comparison frequency, as the data grows faster and faster, the number of splits will also increase, and splits are time-consuming. resource consuming

If the pre-partition is pre-built from the beginning, but because the prefix of the primary key of the index table is often the same consecutively and the order increases, this will still cause a large number of reads, writes or other operations to access one or a few nodes in the cluster. Cause a single machine where a hotspot region is located to exceed its own capacity, causing performance degradation or even region unavailability

Images