Method for implementing transaction processing for real-time full-text search engine

Abstract

The invention discloses a method for supporting transaction processing in a real-time search engine. The method is characterized in that the real-time distributed search engine 130 is a main storage area for storing index data which become effective after operation of an index interface is completed; an index management module 110 is provided with a local index cache 120 organized in transaction units, newly created, modified and deleted data of interfaces of the index management module 110 are stored in the cache temporarily in a transaction process, after index data operation is completed, the data are submitted and conditions for enabling the data to become effective are met, the data, which are relevant to a transaction, in the cache are synchronized to the main storage area of the real-time distributed search engine asynchronously, and accordingly transaction operation for the interfaces is realized. The method has the advantages that the low-cost transaction operation for accessing the real-time search engine is realized, completeness, transaction isolation and correctness of the data applied to updating the search engine in real time are simply and effectively guaranteed, and the original real-time distributed search engine with characteristic of batch processing is unchanged.

Description

technical field [0001] The invention relates to the problem of data integrity in a cloud computing search engine, in particular to a real-time search engine transaction operation method which can effectively solve the problem of data integrity when a real-time application updates the data of the search engine. Background technique [0002] Full-text retrieval means that the computer index program builds an index for each word by scanning each word in the article, indicating the number and position of the word in the article. Search, and feed back the results of the search to the user's retrieval method. This process is similar to the process of looking up a word through a search word table in a dictionary. [0003] The full-text search engine is a software system established according to the full-text search theory to provide full-text search services. Generally speaking, full-text retrieval needs to have the basic functions of indexing and querying. In addition, modern fu...

AI Technical Summary

Problems solved by technology

[0005] The current full-text search engines like Solr have certain defects in ensuring data integrity. All index modification operations are basically based on a single document, so generally only the atomic transaction operation of a single document can be guaranteed. In a transaction, if When multiple document read and write operations are involved, if an error occurs in a certain step of the process (possibly caused by modifying the index itself or modifying the database records), it is impossible to perform transactional recovery on the document index that has been written like a traditional database. Roll operation, the defect of this kind of transaction implementation cannot satisfy those applications with high data integrity requirements

Images