New word discovery method and device

Abstract

A method and device for discovering new words, said method comprising: preprocessing received corpus to obtain text data; performing line-by-line processing on said text data to obtain sentence data; The sentence data is subjected to word segmentation processing to obtain word data after word segmentation; adjacent word data after word segmentation is combined to generate candidate data strings; judging whether the candidate data strings are specific candidate data strings, The specific candidate data string includes basic nouns, and the words at specific relative positions of the basic nouns are nouns or adjectives; judging the candidate data strings to find new words. The method and device can improve the accuracy rate of new word discovery.

Description

technical field [0001] The invention relates to the field of intelligent interaction, in particular to a new word discovery method and device. Background technique [0002] In many fields of Chinese information processing, it is necessary to complete corresponding functions based on dictionaries. For example, in an intelligent retrieval system or an intelligent dialogue system, through word segmentation, question retrieval, similarity matching, determination of retrieval results or intelligent dialogue answers, etc., each process is calculated by using words as the smallest unit, and the basis of calculation is Word dictionary, so the word dictionary has a great impact on the performance of the entire system. [0003] The progress and changes of social culture and the rapid development of economy and commerce often drive the change of language, and the most rapid manifestation of language change is the emergence of new words. Especially in a specific field, whether the wor...

AI Technical Summary

Problems solved by technology

[0005] The accuracy of new word discovery in the existing technology needs to be improved

Images