Method and device for establishing index

Abstract

The invention discloses a method and a device for establishing an index. The main technical scheme is that the method comprises the following steps of: monitoring the amount of indexing segments stored in a first storage area and comprising single documents; when the amount of the indexing segments comprising the signal documents reaches a first threshold value, combining the indexing segments comprising the signal documents into an indexing segment comprising the first threshold value; monitoring the amount of the indexing segments comprising first threshold value files in a first storage area; and when the amount of the indexing segments comprising the first threshold value files reaches a second threshold value, combining the indexing segments comprising the first threshold value files into an indexing segment comprising files multiplying the first threshold value by the second threshold value, and when the total amount of the files included in all the indexing segments in the first storage area reaches a set maximum combination threshold value, combining all the indexing segments in the first storage area into an indexing segment comprising maximum combination threshold value files, and writing the indexing segment into a second storage area. By adopting the technical scheme, the times for disc writing are reduced, the index updating efficiency is guaranteed, and the retrieval efficiency is improved.

Description

technical field [0001] The invention relates to the field of information retrieval, in particular to an index establishment method and device. Background technique [0002] The core technology of a retrieval engine is an index, which is a sequence including several documents. Based on the rapid update of information content, the sequence of documents included in the index also needs to be continuously updated. Currently, the problem with index updates is that in order to update a small number of documents, the entire index needs to be rewritten, but in fact the vast majority of documents in the index are irrelevant to this update. Therefore, in order to reduce unnecessary updates, a segmentation mechanism is commonly used when indexing, that is, the number of documents included in the index is preset, and the index is divided into multiple sub-indexes, and each sub-index is called an index segment (segment). The segmentation mechanism solves the problem of incremental upda...

AI Technical Summary

Problems solved by technology

Therefore, if there are too many index segments in the index, there will be more index files to be opened, read, and processed during retrieval, which will affect retrieval efficiency

However, if the number of index segments in the index is too small, which means that the capacity of the index segment is large, then when the index is updated, it is necessary to perform multiple write operations to the disk, which will inevitably affect the efficiency of the index update

[0008] To sum up, the number of index segments in the index directly affects the efficiency of retrieval and index update. If the number of index segments in the index is too large, it will affect the efficiency of retrieval; if the number of index segments in the index is too small, it will affect the efficiency of indexing. update efficiency

Images