Combination optimizing method based on Lucene index section

Abstract

The invention relates to a combination optimizing method based on a Lucene index section, and belongs to the technical field of the computer index. The method comprises the following steps: combiningcurrent node load information and section information of index, building a combination analyzing module to judge whether to meet a combination condition or not; according to a dictionary file contained in each index section, to obtain a characteristic matrix in the index with respect to an index section, processing by combining a minHash algorithm and a minimum hash signature algorithm, so as to calculate the signature matrix of the index section; through combining the signature matrix of the index section and a Jaccard similarity principle, calculating a similarity coefficient between the index sections, and according to the similarity coefficient, dividing the index sections into different similar sets; and using a similarity evaluation model to grade each similar set, and sorting according to a set score, selecting one or more sets with the highest score to be combined by a combination thread. The optimizing method is capable of reducing the effect of combination operation to performance of an index function and a search function and effectively improving a search speed.

Description

technical field [0001] The invention belongs to the technical field of computer indexing and relates to a method for merging and optimizing based on Lucene index segments. Background technique [0002] Lucene is not a full-text indexing application that can be used directly, but a full-text indexing toolkit written based on the java development language. It is convenient and quick for Lucene developers to develop full-text indexing applications. [0003] There are several indexes in Lucene, and there are several index segments under each index. The index segment is composed of inverted files, forward files and some intermediate files, while the inverted files are composed of dictionaries and posting lists. When new data is added to the index, the data will first be written to the cache in Lucene, and when the cache reaches a certain threshold, it will be flushed to disk to form a new index segment. Each index segment is an independent retrieval unit and can be retrieved. ...

AI Technical Summary

Problems solved by technology

[0006] In view of this, the purpose of the present invention is to provide a method for merging optimization based on Lucene index segments, which is used to solve the problem that the index segment merging process affects the performance of index functions and retrieval functions, and the selected index segments to be merged cannot improve retrieval speed. question

Images