Unambiguous Japanese name list building method and name identification method and device

Abstract

The invention discloses an unambiguous Japanese name list building method and a name identification method and device. The list building method comprises the following steps: performing name separation on a Japanese common name list by using a Japanese surname list to obtain a Japanese name list; dividing training corpuses with Japanese name relevant marks into a Japanese name relevant word set and other word sets; combining the Japanese name relevant word set, the Japanese common name list, the Japanese surname list and the Japanese name list obtained by performing name separation on the Japanese common name list into a Japanese name relevant word total set; specific to each Japanese name relevant word in the Japanese name relevant word total set, judging whether the word is an unambiguous Japanese name relevant word to establish an unambiguous Japanese name relevant word list. Through word segmentation and name role marking by using the list, the overall accuracy of Chinese word segmentation can be increased, and the overall name role marking performance and the final name identification result are enhanced.

Description

technical field [0001] The invention relates to the field of text recognition, in particular to a method and device for recognizing Japanese names. Background technique [0002] With the development of communication technology, exchanges among countries have become increasingly extensive, and information dissemination has become more convenient and faster. Therefore, Chinese texts will contain a large number of named entities such as foreign institution names, person names, and place names. However, these named entities themselves do not necessarily exist in traditional dictionaries, and belong to unregistered words (Out Of Vocabulary, OOV for short), which brings difficulties to many natural language processing-related applications based on Chinese word segmentation. . Effective identification of these named entities can effectively improve the application effects of network text classification, entity association network construction, and topic detection and tracking. ...

AI Technical Summary

Problems solved by technology

However, this method has many limitations. First of all, the discovery of Japanese surnames is very important for the surnames such as "Lin" and "Yin" that exist in both China and Japan, or "Shuxia" and "Datong" that have very large differences in Chinese texts. Ambiguous Japanese surnames will produce wrong recognition results; secondly, the suffixes of personal names often do not appear after Japanese personal names, so the coverage of the recognition method after the personal name is not very high; in addition, because some charact

Images