Search text folding processing system and method

Abstract

The invention relates to a search text folding processing system and method. The system comprises a fingerprint creation module, a database and a folding module; the fingerprint creation module is connected with a local storage end and used for creating document fingerprints corresponding to texts of the local storage end; the fingerprint creation module is connected with the database so as to store the document fingerprint and the corresponding text in the database; the input end and the output end of the folding module are connected with the database and the search engine end respectively, the database is connected with the search engine end, the database outputs a search text and a corresponding document fingerprint according to a search instruction of the search engine end, and the folding module folds the search text according to the document fingerprint to obtain folded text data, and transmits the folded text data to a search engine end so as to be displayed on a search page. Compared with the prior art, the method and the device have the advantages that the same or high-similarity search texts can be folded, so that a user can conveniently view different search texts on thesame page as much as possible.

Description

technical field [0001] The invention relates to the technical field of text analysis, in particular to a search text folding processing system and method thereof. Background technique [0002] At present, when users obtain search texts from local search engines, there are often many duplicate texts with the same or high similarity, so that users cannot obtain all search texts comprehensively and quickly. See the different text for the search. [0003] To this end, the existing technology calculates the similarity of different search texts to facilitate users to know in advance whether there are duplicate texts with the same or high similarity among the search texts. At present, most of them use the calculation of edit distance and the Jaccard coefficient Calculation, TF calculation, word2vec and other methods to calculate the similarity, among which, the edit distance calculation refers to the minimum number of editing operations required to convert one string into another ...

AI Technical Summary

Problems solved by technology

The above method is only suitable for a small number of short text similarity judgment calculations. When faced with a large number of long text similarity judgment calculations, the efficiency of the above methods is relatively low

In addition, although the similarity calculation can enable users to know whether the search texts are the same or similar, it still requires the user to pull down or turn pages multiple times to further view more search texts

Images