High-availability distribution type full-text index method

Abstract

The invention discloses a high-availability distribution type full-text index method. The method comprises the following steps of: firstly, starting a basic service system, and then starting an index cluster service and an inquiry cluster service on each node; establishing, updating and deleting an index on full-text data; and finally, inquiring the index. According to the high-availability distribution type full-text index method, an inquiring and indexing process can be separated and the increment or batch type establishment of existing indexes can be simultaneously supported; the batch type indexes can be used for establishing an index for large-scale data in short time; and the increment type indexes avoid reestablishing the index. Index files can be divided into three layers of structures comprising an index file, an index fragment and an index sub-fragment, thus enhancing the expandability and the availability of the index file. According to the high-availability distribution type full-text index method, a dynamic index task configuration task is provided; and parameters in the index task are arranged to dynamically meet requirements of establishing the index by a user through different types of data.

Description

technical field [0001] The invention relates to the field of information indexing and searching, and more specifically, relates to a method for constructing a distributed full-text index for massive text data and providing highly available query services. Background technique [0002] With the development of the Internet, especially the emergence of Web2.0, the amount of text information is increasing exponentially. Users want to be able to effectively manage massive text data, and quickly search these texts to obtain corresponding information. [0003] The emergence of search engines such as Google, Baidu, and Bing has met the needs of users for information search. The core technology used by search engines is to collect various data information in the network through web crawlers, index these data, and then provide query services to the outside world. As the data information continues to grow, the size of the created index files also increases, resulting in the stand-alo...

AI Technical Summary

Problems solved by technology

In addition, some other distributed index systems divide the index according to a fixed size. During the query process, all index fragments need to be queried, which increases the query time and network overhead.

[0006] 3. Generally speaking, distributed indexing systems are designed to meet specific needs and cannot support dynamic indexing operations on different types of data

When a node in the index cluster fails, use the backups of other nodes in the system to restore it. However, if these backup nodes fail at the same time, the index on the failed node cannot be restored normally.

Images