Intelligent full-text retrieval method and system based on semantic understanding

Abstract

The invention discloses an intelligent full-text retrieval method and system based on semantic comprehension, and the method comprises the steps: segmenting a received search statement into short texts, and carrying out the word segmentation operation of the short texts, so as to obtain a word segmentation library corresponding to the short texts; constructing a semantic information vector and a dependency relationship vector of the short text, wherein the semantic information vector comprises a head word and a word-meaning co-occurrence word of the short text; and based on the semantic information vector and the dependency relationship vector of the short text, performing similarity calculation on the short text information and related information in the intelligent index database to obtain a search result set. According to the method and system, the original data is divided into the multiple short texts to form the search text vector, and the similarity between the search text and the index database text is calculated by calling the semantic understanding interface of the artificial intelligence platform, so that the accuracy of full-text retrieval can be improved.

Description

technical field [0001] The invention relates to the technical field of natural language processing, in particular to an intelligent full-text retrieval method and system based on semantic understanding. Background technique [0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art. [0003] Full-text retrieval takes all kinds of data, such as text, sound, image, etc. as the processing object, and provides a means to realize information retrieval according to the content of the data instead of the external characteristics. It includes two functions: data management and data query, Help users quickly manage and retrieve a large number of documents. [0004] Lucene is currently an open source project of Apache Corporation, and it is also the most popular Java-based open source search toolkit for the entire network. Lucene implements some common word segmentation algorithms and res...

AI Technical Summary

Problems solved by technology

The main reason is that the traditional full-text search divides the original data into words through word segmentation, and links the keywords with all the documents containing these keywords through the inverted index method. When users search, they often just quickly find The documents containing the search keywords are returned, which is only a mechanical match from the font, so a lot of information that represents the same concept but different text expressions will be missed, that is, the keywords cannot be understood semantically

For example, "Cities where four seasons are like spring", users want to get Kunming, Xiamen, Dali and other cities, but the traditional full-text search will only match articles with keywords such as "four seasons" and "city" based on keywords, which is difficult to satisfy users. real needs

[0006] In addition, most of the search fields for full-text retrieval are short texts. The uniqueness of short text information makes its classification method different from the traditional long text classification process. Scholars face the data sparsity, high latitude, semantic A series of studies have been carried out on problems such as lack of information; the existing technology applies the Deep Neural Network (DNN) method to the classification of short texts, and has achieved certain results, but it still faces some challenges. For example, most short text classification models only consider the literal meaning, which is not effective for the recognition of common polysemous words, and cannot solve the defect of short text sparsity

Images