Part-of-speech tagging model training device and part-of-speech tagging system and method thereof

Abstract

The invention relates to a part-of-speech tagging model training device, which comprises a direct constituent analysis unit for performing direct constituent analysis of words to obtain direct constituents, attributes thereof and position relations thereof; a converting unit for converting the results of the direct constituent analysis into training data; and a machine learning unit for machine leaning of the converted training data to generate a part-of-speech tagging model. The invention relates to a part-of-speech tagging model training method and a part-of-speech tagging system and a method thereof. The system comprises the part-of-speech tagging model training device for performing the direct constituent analysis of words in a dictionary to generate the part-of-speech tagging model and a part-of-speech tagging device based on the model for tagging unlisted words by using the part-of-speech tagging model. According to the system of the invention, the part of speech of the unlistedwords can be tagged accurately on the basis of the prior text message, and the efficiency for the text message processing is improved.

Description

technical field [0001] The present invention relates to the field of text information processing, in particular to a part-of-speech tagging model training device and method thereof, and a part-of-speech tagging system and method thereof. Background technique [0002] With the widespread popularization of the Internet and the increasing informatization of society, there are more and more text information, and the corresponding social demand for text information processing is increasing. People are increasingly eager to use natural language to communicate with computers, and hope to use Automated means to process massive text information. In order to process text information better, people need to accumulate a large amount of language data resources, including dictionaries. As an important tool for processing texts - dictionaries are often compiled by humans. The main components of dictionaries include words and their attributes. The currently developed unregistered words (ma...

AI Technical Summary

Problems solved by technology

[0009] The current technology does not effectively solve the problem of automatic part-of-speech tagging of unregistered words. Among them, the patent CN1369877 cannot give a reasonable part-of-speech judgment for unregistered words with zero character separation probability. In addition, the accuracy of the part-of-speech tagging depends on the selected Dictionary, while the method Recall (20%) corresponding to the literature [Lu, X.F.Hybrid Methods for POS Guessing of Chinese Unknown Words.Proceedings of the ACL Student Research Workshop, pages 1-6] is relatively low

Images