HBase big data-based real-time full-text retrieval system and realization method for same

Abstract

The invention provides an HBase big data-based real-time full-text retrieval system and a realization method for the same. The system comprises an HBase cluster copying module, started with an HBase cluster at the same time to monitor whether a RegionServer of a slave cluster is registered in real time, and synchronizing data change to the registered RegionServer when the data change of the HBasecluster is monitored, an index server module realizing an interface of the RegionServer to simulate a RegionServer of one slave cluster, including steps of calling a ReplicationAdmin interface of theHBase to register in the HBase cluster copying module, informing the HBase cluster of the RegionServer of the slave cluster, receiving data transmitted by the HBase cluster, and converting the data according to a search engine format and transmitting the data to a search engine module after the data of the HBase cluster is received, and the search engine module establishing a full-text index according to the converted data to realize real-time full-text retrieval to the HBase data for a user. Real-time full-text retrieval with high efficiency and low time delay can be realized with HBase working as data storage.

Description

technical field [0001] The invention relates to the technical field of computer processing, in particular to a real-time full-text retrieval system based on HBase big data and an implementation method thereof. Background technique [0002] As a NoSQL database, HBase is increasingly used in online production environments to store massive amounts of data. HBase is a key-value storage and column family-oriented database, which can quickly locate the corresponding row according to the key-value key that needs to be queried. However, in many scenarios, users often only know the value to be queried, but not the key corresponding to the value. Even if the user only knows a certain part of the value, like a search engine, it can retrieve all the rows containing only part of the value in full text. At this time, HBase can only scan the entire table for this requirement, and retrieve the values ​​of all rows. Then check whether this value has the content that the user needs to query...

AI Technical Summary

Problems solved by technology

However, in many scenarios, users often only know the value to be queried, but not the key corresponding to the value

Even if the user only knows a certain part of the value, like a search engine, it can retrieve all the rows containing only part of the value in full text. At this time, HBase can only scan the entire table for this requirement, and retrieve the values ​​of all rows. Then check whether this value has the content that the user needs to query. This method is inefficient, and as the number of HBase storage becomes larger and larger, the delay is very serious and cannot meet the user's needs at all.

Images