A file indexing system and method based on elasticsearch full-text retrieval

Abstract

The invention discloses a file indexing system and method based on ElasticSearch full-text retrieval, wherein the system includes a user operation management module, a file system management module, a file system, an ElasticSearch cluster module, a database system, and an ElasticSearch client module. The retrieval will be completely real-time, and the retrieval results at any time will not be outdated or wrong, and at the same time effectively reduce the waste of resources caused by ElasticSearch when performing full-text retrieval of the file system; and the method described can realize bidirectional communication between files and ElasticSearch Exchange, support the acquisition of the path of the file where the search result is located, and perform more operations on the hit file.

Description

technical field [0001] The invention relates to a software retrieval system and method, in particular to a file indexing system and method based on ElasticSearch full-text retrieval. Background technique [0002] At present, the two mainstream full-text search tools in the world are solr and Elasticsearch. They both use the Lucene framework as the core of the search engine, but they target different scenarios. Solr is mainly used in scenarios with diverse data formats and low data content update frequency, while Elasticsearch applications In scenarios where the data format is single and the data content is updated frequently, Elasticsearch is relatively slower than Solr to query the index, but the index creation speed is significantly higher than that of Solr. For the scenario where the file system often has frequent updates, Elasticsearch can be used. To achieve the purpose of quickly updating the index, but because Elasticsearch only supports data input in json format, and...

AI Technical Summary

Problems solved by technology

[0002] At present, the two mainstream full-text search tools in the world are solr and Elasticsearch. They both use the Lucene framework as the core of the search engine, but they target different scenarios. Solr is mainly used in scenarios with diverse data formats and low data content update frequency, while Elasticsearch applications In scenarios where the data format is single and the data content is updated frequently, Elasticsearch is relatively slower than Solr to query the index, but the index creation speed is significantly higher than that of Solr. For the scenario where the file system often has frequent updates, Elasticsearch can be used. To achieve the purpose of quickly updating the index, but because Elasticsearch only supports data input in json format, and the index field is not allowed to be modified after the index is created, this brings some application troubles to the application of the index system for the file system. Currently, the application There are still many deficiencies in the technology:

[0003] 1. At present, the full-text search tool ElasticSearch is used for the index query application for the file system, which can only index and display the content of the file, and cannot feed back the path information of the file to the file system management program, so that the management program can perform various operations on the files to which the search results belong. a management operation;

[0004] 2. At present, the full-text search tool ElasticSearch is used to index and query the file system in quasi-real-time or non-real-time, and it cannot be real-time. That is, when there is a file update in the file system, the search result immediately reflects this update, and always keeps the two synchronized and consistent sex;

[0005] 3. At present, there are multiple data copies when using the full-text search tool ElasticSearch to index and query the file system, resulting in a waste of storage resources

Images