Implementation method of HBase secondary index

An implementation method, a secondary index technology, applied in database indexing, structured data retrieval, digital data information retrieval, etc., can solve problems such as unsuitable complex logic query, resource consumption, etc., to improve query and retrieval efficiency, efficient query and retrieval function, the effect of saving memory cache space and disk storage space

Pending Publication Date: 2021-05-14
SHANDONG LANGCHAO YUNTOU INFORMATION TECH CO LTD
View PDF6 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

HBase is based on the rowkey primary key query speed in milliseconds, but HBase is not suitable for complex logic queries, which often require full table scanning, which consumes a lot of resources

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Implementation method of HBase secondary index
  • Implementation method of HBase secondary index

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0030] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Apparently, the described embodiments are only some of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.

[0031] The NiFi data streaming platform is an easy-to-use, powerful, and reliable data processing and distribution system. Based on the Web graphical interface, complete the process-based programming through dragging, connecting, and configuring to realize data collection and other functions. Suitable for visual creation and management of directed gr...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention particularly relates to an implementation method of HBase secondary index. According to the implementation method of the HBase secondary index, an Elasticsearch search engine, a NiFi data flow platform and an HBase distributed column storage database are integrated; wherein the NiFi data flow platform is responsible for extracting source data and writing the source data into the Elasticsearch engine and the HBase distributed column storage database, the Elasticsearch engine is responsible for storing indexed data and a rowkey primary key of the HBase distributed column storage database, and the HBase distributed column storage database is responsible for storing full detailed data; then an Elasticsearch engine, according to the query condition, searches a rowkey primary key of the HBase distributed column storage database, and the detailed data stored in the HBase distributed column storage database is queryed by using the rowkey primary key as the query condition, so that an efficient query retrieval function is provided for the HBase. According to the implementation method of the HBase secondary index, the memory cache space and the disk storage space of the server can be saved, efficient query and retrieval functions are provided for the HBase, and the query and retrieval efficiency is greatly improved.

Description

technical field [0001] The invention relates to the technical field of data retrieval, in particular to a method for realizing an HBase secondary index. Background technique [0002] With the rapid development of computer technology and network technology, massive data is stored in the HBase database. In the HBase database, only the Rowkey primary key is used as the first-level index. If you want to perform data retrieval and query on the non-primary key fields of HBase, you often need to scan the entire table through a distributed computing framework such as MapReduce / Spark, and the hardware resource consumption and time delay will be relatively high. . HBase cannot satisfy the fast and complex query function of data. The advantages and disadvantages of HBase data storage are as follows: [0003] Apache HBase is a Hadoop database, a distributed, scalable, big data storage database. The HBase distributed column store database can host very large tables on commodity hardw...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/22G06F16/2453G06F16/215G06F16/27
CPCG06F16/2228G06F16/2453G06F16/27G06F16/215
Inventor 赵圣杰徐伟涛高传集胡清
Owner SHANDONG LANGCHAO YUNTOU INFORMATION TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap