Text Analysis Method and Text Analyzer

Abstract

The invention discloses a text analysis method and a text analyzer. The method comprises the following steps of: performing splitting processing on an acquired text by utilizing characters as a unit, and performing characteristic tagging on characters obtained by splitting according to preset character characteristics so as to form tagged word strings; performing word segmentation processing on the tagged word strings according to pre-constructed word segmentation models so as to obtain word segmentation results containing word orders; performing merging processing on the word orders contained in the word segmentation results, and performing characteristic tagging on words obtained by merging according to the preset character characteristics so as to obtain tagged word strings; performing part-of-speech tagging on the tagged word strings according to pre-constructed part-of-speech tagging models so as to obtain part-of-speech tagging results; and if confirming that the part-of-speech tagging results contain part-of-speech tags of entity words, merging the entity words containing the part-of-speech tags in the part-of-speech tagging results according to same adjacent rules, so as to obtain a text analysis result. By applying the text analysis method and the text analyzer, the entity word text analysis accuracy rate can be improved.

Description

technical field [0001] The invention relates to data mining technology, in particular to a text analysis method and a text analyzer. Background technique [0002] At present, in natural language processing technology, lexical analysis of text is the basis of other Chinese information processing. For example, the widely used search engines, machine translation, speech synthesis, automatic classification, automatic summarization, automatic proofreading, etc. Lexical analysis techniques. There are two purposes for lexical analysis of a sentence or text: word segmentation and part-of-speech tagging. Word segmentation is to divide the text sequence closely connected between words into words, so as to convert the text sequence into a word sequence; Basically, according to the context information of the sentence, the part-of-speech tagging is performed on the divided words, for example, the words are marked as verbs, nouns, adverbs, or adjectives. Among them, words are the smalle...

AI Technical Summary

Problems solved by technology

[0006] It can be seen from the above that the existing text analysis methods use a unified strategy to analyze the text without distinguishing between entity words and non-entity words, which makes the text analysis accuracy of entity words low

Images