HBase full-text index asynchronous construction method

A full-text indexing and asynchronous technology, applied in special data processing applications, instruments, electrical digital data processing, etc., to achieve the effect of avoiding storage and computing overhead and good timeliness

Active Publication Date: 2017-05-31
SHANDONG LANGCHAO YUNTOU INFORMATION TECH CO LTD
View PDF3 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Solve the problem of real-time full-text indexing of HBase data, avoiding additional storage and computing overhead

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0021] The content of the present invention is described in more detail below:

[0022] In order to ensure the reliability of the data, the data writing process of HBase first writes the data into the WAL log. The WAL log only records two operations, write and delete. The WAL log is rolled over periodically.

[0023] The process of HBase asynchronously building a full-text index for data is as follows:

[0024] 1. Register a peer cluster (another HBase cluster) in the HBase cluster, and set the peer cluster on the HBase table Table_Index that needs to establish a full-text index.

[0025] 2. In Zookeeper, a queue is maintained for each RegionServer, and the WAL logs that need to be read are stored in the queue.

[0026] 3. HBase starts a separate thread on the RegionServer where the Table_Index table is located, reads the WAL log according to the WAL log queue in Zookeeper, and parses it to analyze which data is related to the Table_Index table.

[0027] 4. Delete the read ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention belongs to the field of computer software application and provides an HBase full-text index asynchronous construction method. According to the method, HBase data in big data scenarios are subjected to asynchronous real-time construction of full-text indexes; by reading and analysis of HBase WAL logs, the full-text indexes of the data are constructed according to configurations of users. Therefore, storage and computation cost caused by extra data processing is avoided.

Description

technical field [0001] The invention relates to computer software application technology, in particular to a method for asynchronously constructing an HBase full-text index. Background technique [0002] With the continuous development of cloud computing technology, cloud computing technology has become an important pillar supporting the development of information technology in various industries. Distributed clusters based on Hadoop and HBase have become a popular research object of cloud computing at home and abroad. Hadoop's HDFS distributed storage provides a distributed file storage system for the cloud platform, and HBase has good read and write performance and can support tables with large amounts of data, so it is suitable for simple business and online databases with huge data volumes and data storehouse. Since HBase has relatively weak support for transactions and only supports row-level transactions, the business database is often served by mature relational dat...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/2228G06F16/2358
Inventor 臧勇真
Owner SHANDONG LANGCHAO YUNTOU INFORMATION TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap