Lightweight text fuzzy search method supporting semantic association

Abstract

The invention discloses a lightweight text fuzzy search method supporting semantic association. According to the method, a traditional statement retrieval algorithm is improved, statements which are completely consistent with target statements and are high in similarity can be retrieved, and approximate values of the statements and the target statements can be flexibly adjusted; the operation speed is high: a traditional violent enumeration algorithm is abandoned, and methods such as semantic atlas, convolution and dynamic programming are used, so that the search process is optimized, and thesearch speed is greatly increased and the size of the system is reduced, internal and external optimization is performed for lightweight users and use scenes, the whole calculation process is optimized, and the memory burden is reduced. The invention further provides a set of association mode without field operation, a user can call the association module in fuzzy search, but does not need to occupy the local computing power; the system is flexible, a user can easily and flexibly call different applications, and the whole algorithm module is subjected to interface packaging.

Description

technical field [0001] The invention relates to the related field of text fuzzy search, in particular to a light-weight text fuzzy search method supporting semantic association. Background technique [0002] Text fuzzy search is applied in many places, especially nowadays the network is increasingly developed, and the amount of text information generated on the network is also growing explosively. As a result, harmful information and information causing instability are increasingly flooded. Therefore, on public network platforms, many contents need to be censored before they can be displayed. In the early days of Internet censorship, most of them were reviewed manually, which was very inefficient, and compared with the speed of Internet text generation, this efficiency was even more insignificant. Therefore, many scholars and companies pay more attention to the fuzzy search of text, that is, to fuzzily find a given keyword or key sentence in a large amount of text informati...

AI Technical Summary

Problems solved by technology

[0004]1. Most of the current fuzzy text searches do not reflect the real fuzzy search well. Simply speaking, the degree of fuzziness is relatively low and cannot be well supported Semantic association, such as searching for synonyms and associated words of keywords, so the synonyms of keywords will be filtered out, but in actual applications, it may need to be retained, which causes false filtering and lowers the recall rate

Moreover, when searching for keywords or key sentences in relatively long texts, due to the use of more violent methods to process the texts, the efficiency is relatively low, that is to say, it is not lightweight enough;

[0005]2. Most of the current text fuzzy search has not solved the two main problems of character string fuzzy matching: space problem and time problem. With a large amount of calculation and storage, the existing fuzzy matching algorithms often cannot meet the actual online needs in terms of time complexity and space complexity;

[0006]3. Most of the current text fuzzy search cannot capture the sentence-level features. If there is no text to be searched in the searched text, the search result is empty

However, there may be texts with similar meanings to the texts that need to be searched. In practical applications, if this happens, the search results are often not expected to be empty, but the texts with similar meanings are returned as results.

Images