A method for grading Chinese texts and calculating difficulty scores for Chinese texts

Abstract

The invention relates to a method for grading Chinese texts and calculating the difficulty score of Chinese texts. Firstly, text acquisition and grade labeling are performed to obtain articles with classification labels as a training set, and then feature extraction is performed to obtain the text corresponding to each article. The eigenvalues ​​of all the linguistic features of the text; then, the establishment and testing of the model are carried out, and the prediction accuracy reaches the expected model, and finally the obtained model is used to predict the difficulty of the text. The present invention is applicable to various scenarios where text legibility needs to be evaluated, and the adopted support vector regression algorithm model achieves a better scoring effect by increasing the dimensional space of features, which is superior to traditional linear models.

Description

technical field [0001] The invention relates to the technical field of model prediction, in particular to a method for grading Chinese texts and calculating the difficulty score of Chinese texts. Background technique [0002] With the development of network technology, massive amounts of unstructured data such as text, images, and videos are generated on the Internet every day. The text data in it can be processed by modern natural language processing technology to dig out more valuable information hidden in the text. For a long time, the mainstream technology of natural language processing was rule-based, from various syntactic analysis to semantic analysis, and then with the development of the Internet, accompanied by the generation of a large amount of corpus, statistical natural language processing gradually emerged . Text legibility research is one of the research points. In the process of language learning, teachers need to select moderately difficult texts from a l...

AI Technical Summary

Problems solved by technology

However, if it is extended to a more general scenario, there will be problems such as the selected features are not comprehensive enough, and the linear model used is not ideal.

More importantly, the features selected for Pinyin texts cannot well reflect the difficulty characteristics of Chinese texts

Since there are no tools in China that can score the difficulty of Chinese texts, many domestic text difficulty scoring tasks, such as teaching material ratings, are still done using the most primitive manual grading.

However, for the existing Chinese texts with various themes and different styles, manual difficulty rating is a huge workload and very time-consuming.

Moreover, the results of manual grading are often very subjective, and it is difficult to reach a consensus during retesting

Images