Synonym screening method and system

Abstract

The invention provides a synonym screening method which includes the steps: training the word vector of a big corpus word; mining synonyms of the big corpus word, and particularly, acquiring a candidate synonym set; updating synonym similarity; performing screening to obtain a synonym list. Compared with the prior art, the synonym screening method, the synonyms obtained by big corpus training are wide in coverage, synonyms with good timeliness can be found by adding new big corpuses, the synonyms obtained by screening according to the principle of the requirement of the synonyms for synonymy are higher in quality, and a forceful tool for natural language processing semantic comprehension is added. The synonym screening method is applied to a chatting robot, sentences expressing the same meaning with different words by a user can be more effectively recognized, and the sentence understanding level of the robot is improved.

Description

technical field [0001] The invention relates to the field of artificial intelligence, in particular to a method and system for screening synonyms. Background technique [0002] In the design of chat robots, it is often necessary for the computer to understand the same sentence of the user and use different expressions to improve the robot's recognition level of the sentence, among which the conversion of synonyms is the most common method. Synonyms play an important role in basic applications such as information extraction, question answering systems, and data mining. Existing methods for mining synonyms either have narrow coverage of words, or the acquired synonyms are relatively old, or the quality of synonyms is not high. These problems affect the application of synonyms in the field of natural language processing. [0003] The methods adopted in prior art for mining synonyms mainly include: [0004] 1. Rely on the rule method of ontology dictionary or knowledge base. ...

AI Technical Summary

Problems solved by technology

[0008] 1. For rule methods that rely on ontology dictionaries or knowledge bases, since dictionaries and knowledge bases mostly rely on manual construction, their timeliness and coverage are relatively poor

[0009] 2. The method based on search log behavior needs to use the structural template of the synset, and the scalability and coverage are not good

[0010] 3. The cosine similarity expressed by the word vectorization of the neural network language model is used to measure the semantic similarity of words. This method has a certain effect, but the existing methods cannot obtain high-quality synonyms

The word vector of the neural network language model can reflect the semantic similarity to a certain extent, but some of the obtained similar words are not semantically similar, and these methods cannot effectively remove non-synonymous words to obtain high-quality synonyms

[0011] To sum up, the existing methods for obtaining synonyms cannot meet the requirements of wide coverage, good timeliness, and high quality at the same time, and cannot meet the needs of natural language processing, and it is also difficult to improve the level of sentence understanding of chat robots

Images