Linear model method used for simplified-Chinese readability measurement

Abstract

The invention discloses a linear model method used for simplified-Chinese readability measurement. The method includes the steps of: constructing simplified-Chinese text and a readability level corpusthereof; preprocessing the text, wherein word segmentation, sentence segmentation, part-of-speech labeling, named-entity recognition, component syntax analysis, dependency syntax analysis, clause labeling and stroke counting are included; extracting and calculating text language features; constructing a best feature combination according to the language features and a regression algorithm; and constructing a linear regression model of readability measurement. The text language features adopted by the model cover four aspects of shallow-layer features, part-of-speech label features (also called semantic or lexical features), grammatical features, textual features and the like, a readability level of simplified-Chinese text for learners of which native languages are Chinese can be automatically predicted, and a gap of readability prediction models based on the simplified-Chinese text is filled. The model of the invention is high in a fitting degree, is high in interpretability, and hasextensibility and an important reference value for evaluating readability of application text.

Description

technical field [0001] The invention relates to the technical field of readability measurement, in particular to a linear model method for readability measurement of Simplified Chinese. Background technique [0002] Linguistic complexity is a multidimensional and interdisciplinary academic concept that can be studied from the perspectives of natural language processing, second language acquisition, psycholinguistics, cognitive science, and contrastive linguistics. The definition of language complexity can be carried out from two aspects: language complexity in the strict sense, that is, the research on the complexity of language structure, which is mostly used in cross-language comparative research and automatic scoring of compositions; research on language complexity in a relative sense, such as readability, Language difficulty, cognitive cost, etc., are mostly used in applied research that serves language learning and text understanding. This project examines the linguist...

AI Technical Summary

Problems solved by technology

But there are still many deficiencies, such as the scale of research data is very small (such as twenty or thirty articles or dozens of sentences), over-targeted (such as for foreign students of individual languages), variable selection is relatively subjective (such as omitting automatic learning variables), insufficient empirical tests (such as lack of goodness-of-fit tests), and large differences in the specific design of variable categories among various measurement formulas

Compared with the richness and systematicness of other language readability studies, the empirical research on Chinese readability indicators based on large-scale simplified texts and computational linguistics techniques has not yet been comprehensively and systematically carried out.

Images