Grading treatment method and system based on Lucene fragmentation structure

Abstract

The invention discloses a grading treatment method and system based on a Lucene fragmentation structure. The method comprises the following steps: performing data splitting on an index file to form index file fragmentation data, and distributing the index file fragmentation data on various fragmentation processors, so as to complete initialization operation; receiving query information input by users through a searching processor, and performing word segmentation treatment on the query information to form searching lexical items; looking up in a global information buffer according to the searching lexical items in sequence to judge whether the information related to a current searching lexical item exists or not; if the information related to a current searching lexical item does not exist, directly sending the searching item to the fragmentation processors to be treated; if the information related to a current searching lexical item exists, acquiring the global information of the searching lexical item from the global information buffer by the searching processor, and then sending the searching lexical item to the fragmentation processors to be treated. According to the invention, the global information is treated through multiple requests to enable various fragmentation to share the global information, so that the problem that the computed scores of one lexical item in different fragmentation are different is solved, and the sequence of searching results is more reasonable.

Description

technical field [0001] The invention belongs to the technical field of mass data processing, and in particular relates to a scoring processing method and system based on a Lucene fragmentation structure. Background technique [0002] With the rapid development of the Internet and the rapid growth of Internet information, people have increasingly relied on the Internet to obtain information in their daily work and life, so how to quickly find the information they need is self-evident for people. The traditional relational database retrieval method can no longer support the retrieval of such a large amount of data on the Internet, so full-text search has emerged as a query method with a large amount of data, and the full-text search tool represented by Lucene is based on its The advantages of high efficiency, high accuracy, and high expansion are increasingly used by Internet companies. [0003] However, due to the ease of use of Lucene, it has certain disadvantages when proc...

AI Technical Summary

Problems solved by technology

[0019] Aiming at how to coordinate the scoring among the various fragments in the distributed search, this patent proposes a scoring processing method and system based on the Lucene fragmentation structure, which processes the global information through multiple requests so that each fragment can share the global information and solve the problem of Solved the problem of inconsistent scores calculated by the same term in different shards, making the ranking of search results more reasonable

Images