Uygur language sentence boundary recognition method

Abstract

The invention discloses a Uygur language sentence boundary recognition method. The method includes 1, putting forward recognition rules of unambiguous punctuation marks in Uygur language sentence recognition; 2, putting forward a Uygur language paragraph classification algorithm to effectively reduce scale of statistical space and quickly improve efficiency; 3, using statistics to building Uygur language sentence boundary recognition feature space to efficiently recognize ambiguous punctuation marks in Uygur language sentences; 4, realizing high-performance Uygur language sentence boundary recognition aiming at indiscriminate corpora. By the Uygur language sentence boundary recognition method, accuracy of sentence boundary recognition is effectively improved, and basic analyzing service is provided for natural language processing work like subsequent part-of-speech tagging and syntactic analyzing.

Description

technical field [0001] The invention relates to a language information processing technology, in particular to a Uighur sentence boundary recognition method. Background technique [0002] With the rapid development of Internet technology, all kinds of information are increasing, and massive amounts of information are generated, stored and disseminated on the Internet every day. Human beings are facing unprecedented information expansion. Natural language processing technology is widely used in processing a large number of network information texts. Automatic and efficient Uyghur text analysis technology has become a key technology for information processing and understanding, which is of great importance to the research of language information processing and related application fields. theoretical significance and application value. [0003] The gradual development and maturity of large-scale natural language text acquisition technology, machine learning methods and models,...

AI Technical Summary

Problems solved by technology

However, according to the error amplification principle of natural language processing, the performance of the sentence boundary recognition algorithm at the lowest level in natural language processing directly affects the accuracy of the next analysis, and the impact is relatively large

[0006] Some foreign scholars have established some models and methods of English sentence boundary recognition through long-term research on English features, but these models cannot be directly used in Uyghur sentence boundary recognition tasks, because the two languages ​​produce different ambiguities in sentence boundaries. Different objects for disambiguation and features that contribute to recognition are quite different, etc.

Images