Acquisition system and method of text field based on concept symbols

Abstract

The invention discloses an acquisition system and a method of the text field based on concept symbols. The system comprises a concept symbol set for expressing word concepts and field categories, a word knowledge base for storing word and concept symbols, a word segmentation processor, a statement semantic analyzer and a field arbiter. The method comprises the following steps: (1) segmenting an input text into paragraphs, statements and words; (2) carrying out semantic analysis on the statements for obtaining concept categories and semantic blocks of the statements; (3) obtaining activating words in the statements according to semantic concept symbols in the field concept symbol set and the word knowledge base; (4) carrying out comprehensive scoring on field concept symbols of the activating words and obtaining the field concept symbol with the highest score as the field of the statements; (5) merging the statements in the paragraphs according to the field concept symbols for obtaininga statement group and the field thereof; and (6) obtaining the field of the text according to a title of the text and the frequency of occurrence and the position of the statement group in the statement group.

Description

technical field [0001] The present invention relates to the field of text information processing by using computer science and technology, in particular to a system and method for acquiring text field based on concept symbols. Background technique [0002] Text classification technology is the method and process of using computers to classify a text into one or more domain categories according to certain rules, knowledge and steps. The general method of text classification is to represent texts as feature vectors, and when the "angle" of the feature vectors of two texts is less than a certain angle, they are classified into the same category. Generally, words are selected as the text features to form the feature vector of the text. The construction method of the feature vector mostly uses the TF*IDF method or the TF*IWF method derived from it. TF*IDF uses the frequency of occurrence of words in the document and in the document collection. The product of the reciprocal of th...

AI Technical Summary

Problems solved by technology

However, the domain categories obtained by automatic text clustering are very rough, and the results are difficult to adapt to actual needs due to the lack of classification guidance

Moreover, the same text clustering method has a good effect on a certain text set, but may have a poor effect on another text set, that is, there are shortcomings in the practicality and stability of text clustering.

[0004] In summary, the statistical method of text classification requires a large amount of pre-classified training corpus, which is often difficult to provide during classification.

Although text clustering can overcome this shortcoming, it is difficult to combine the clustering results with the actual needs of classification.

Images