Synonym mining method and device in government affair service field

Abstract

The invention relates to the technical field of artificial intelligence, and provides a synonym mining method and device in the government affair service field, and the method comprises the steps of: generating a new word candidate word set; determining a multivariate statistical index value of each new word candidate word in the new word candidate word set, and screening out the new word candidate words of which the multivariate statistical index values are lower than a preset threshold value to obtain a new word set in the government affair service field; inputting the new word set into a government affair service field lexicon, and constructing an abbreviation synonym set; inputting the abbreviation synonym set into a pre-constructed double-tower model to obtain the similarity between the original entry and the plurality of abbreviation synonym candidate entries; and obtaining abbreviation synonyms corresponding to the original entry according to the similarity between the original entry and the plurality of abbreviation synonym candidate entries, and inputting the abbreviation synonyms into a synonym lexicon in the government affair service field. A new word candidate word set is filtered by using multivariate statistical indexes, so that the accuracy of new word discovery is improved; by constructing the double-tower model, abbreviation synonyms can be effectively recognized, and the synonym mining efficiency is improved.

Description

technical field [0001] The invention relates to the technical field of artificial intelligence, in particular to a method and device for mining synonyms in the field of government affairs services. Background technique [0002] Synonyms are often used in natural language processing tasks such as information retrieval and text classification. Specifically, before processing tasks such as information retrieval or text classification, it is necessary to obtain synonyms and identify synonyms. For example, in the application scenario of information retrieval, multiple words belonging to synonyms can be classified into the same category. When there are keywords with synonyms in the input text, the synonyms can be used instead of the original keywords to search, thereby improving the recall rate of the search. . [0003] However, due to the continuous development of the government service field in recent years, a large number of new vocabulary and proper nouns have emerged, resul...

AI Technical Summary

Problems solved by technology

[0003] However, due to the continuous development of the government service field in recent years, a large number of new vocabulary and proper nouns have emerged, resulting in inaccurate word segmentation and difficulties in mining synonyms

Due to the variety of Chinese semantics, it is difficult to mine synonyms. The existing thesaurus of government service collars is mostly filled manually, which makes the efficiency of synonyms mining low.

Moreover, there is currently no unified standard for the definition of new words in the field of government services; the field of government services covers a wide range of fields, including humanities, economy, society and other sub-fields. It is difficult to grasp the formation rules of new words, and the formation rules of new words in each field are different; However, due to reasons such as sparse text in the field of government services, it is difficult to identify low-frequency new words, and the accuracy of new word discovery is low.

Images