Text readability evaluation method, device, equipment and storage medium

Abstract

The present invention discloses a text readability evaluation method, a text readability evaluation device and a computer-readable storage medium, which include obtaining text data of reading materials; pre-training a general-purpose roberta model according to the text data to obtain pre-training weights; selecting part of the data from the text data for annotation and scoring to construct an annotation data set; building an initial Roberta reading difficulty scoring model, loading the general-purpose roberta model for pre-training and the pre-training weights as initial weights; training the initial Roberta reading difficulty scoring model to obtain a trained Roberta reading difficulty scoring model; inputting the text data of the reading difficulty to be judged into the trained Roberta reading difficulty scoring model for predictive scoring evaluation. The method can help administrators, students, literacy course developers, teachers who select texts and other personnel to quickly and accurately evaluate the reading difficulty of articles.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a text readability evaluation method, a text readability evaluation device and a computer-readable storage medium. Background Art

[0002] At present, most educational texts are matched with readers using traditional readability assessment or commercial calculation formulas. However, there are many problems here. Readability assessment methods are too subjective and inaccurate, while commercial formula calculation methods are too expensive. For example, readability assessment tools such as the Flesch-Kincaid Grade Level use character and syllable information and syntactic complexity in the assessment text to assess difficulty and match it with readers of the corresponding age. These methods lack structured and theoretical proof of effectiveness. In addition, commercial assessment methods such as Lexi le require high prices to purchase. Summary of the invention

[0003] The purpose of the present invention is to solve at least one of the technical problems existing in the prior art, and to provide a text readability evaluation method, a text readability evaluation device and a computer-readable storage medium, which can help administrators, students, literacy course developers, teachers who select texts and other personnel to quickly and accurately evaluate the reading difficulty of an article.

[0004] In a first aspect, an embodiment of the present invention provides a method for evaluating text readability, comprising:

[0005] Get the text data of the reading material;

[0006] Pre-training a general domain roberta model according to the text data to obtain pre-training weights;

[0007] Selecting part of the text data for annotation and scoring to construct an annotation data set;

[0008] Building an initial Roberta reading difficulty scoring model, loading the general field Roberta model for pre-training and using the pre-trained weights as initial weights;

[0009] Using the annotated data set to train the initial Roberta reading difficulty scoring model to obtain a trained Roberta reading difficulty scoring model;

[0010] The text data of the to-be-judged reading difficulty is input into the trained Roberta reading difficulty scoring model for predictive scoring evaluation.

[0011] In the above-mentioned text readability evaluation method, the pre-training of the Roberta model of the general field according to the text data to obtain the pre-training weights includes:

[0012] Load the roberta pre-trained model for general domains;

[0013] The pre-training framework language model is used to pre-train 3 epochs on the text data to obtain and save the pre-training weights.

[0014] In the above-mentioned text readability evaluation method, the annotated data set includes a first-tier data set with annotated scores between 0 and 1, a second-tier data set with annotated scores between 1 and 2, a third-tier data set with annotated scores between 2 and 3, a fourth-tier data set with annotated scores between 3 and 4, and a fifth-tier data set with annotated scores between 4 and 5, wherein the annotated scores are obtained by evaluation and scoring by the evaluator.

[0015] In the above-mentioned text readability evaluation method, the Roberta reading difficulty scoring model is constructed in the following manner:

[0016] Constructing a Roberta-Base model, loading the pre-trained weights of the roberta model in the general field, and adding two fully connected layers and multiple parallel dropout layers to each layer;

[0017] Input the output data of each pre-trained layer into the first fully connected layer, and input the output data of the first fully connected layer into the plurality of dropout layers respectively;

[0018] The output data of multiple dropouts are summed and input into the second fully connected layer, wherein the second fully connected layer is a classification fully connected layer.

[0019] In the above-mentioned text readability evaluation method, the training of the initial Roberta reading difficulty scoring model includes:

[0020] During the training process, different learning rates are set for different pre-training layers. FGM adversarial training is added to perform perturbations on the Embedding layer to increase the robustness of the model. Dynamic padding is used to reduce the amount of model calculations.

[0021] In the above-mentioned text readability evaluation method, in the process of training the initial Roberta reading difficulty scoring model, the loss function of each layer is the sum of the first loss function and the second loss function, where:

[0022] The first loss function is the CrossEntropy loss function:

[0023]

[0024] Where x is the input labeled data set, class is the category to be calculated, and y is the overall category;

[0025] The second loss function is a loss function based on contrastive learning:

[0026]

[0027] Where i represents the positive sample of the labeled data set, j represents the negative sample of the labeled data set, and λ i,j represents the contrast loss of the labeled dataset, z i is the vector representation of data i, z j is the vector representation of data j, z k is the vector representation of the corresponding data k in the negative sample of the labeled data set, N is the number of samples in the labeled data set, and sim() represents calculating the similarity between the positive sample and the negative sample of the labeled data set.

[0028] In the above-mentioned text readability evaluation method, during the process of predictive scoring evaluation, the output classification probabilities of all attention layers are added and averaged to obtain the final output probability, among which the one with the largest probability is used as the output score value.

[0029] In a second aspect, an embodiment of the present invention provides a text readability evaluation device, comprising:

[0030] An acquisition module is used to acquire text data of reading materials;

[0031] A pre-training module, used to pre-train the Roberta model of a general field according to the text data to obtain pre-training weights;

[0032] A marking and scoring module, used for selecting part of the text data for marking and scoring to construct a marked data set;

[0033] A model building module, used to build an initial Roberta reading difficulty scoring model, and load the pre-trained weights of the Roberta model of the general field as initial weights;

[0034] The training and scoring module is used to train the initial Roberta reading difficulty scoring model using the annotated data set to obtain a trained Roberta reading difficulty scoring model; and is used to input the text data of the to-be-determined reading difficulty into the trained Roberta reading difficulty scoring model for predictive scoring evaluation.

[0035] In the above-mentioned text readability evaluation device, the pre-training module pre-trains the Roberta model of the general field according to the text data to obtain the pre-training weights, including:

[0036] Load the roberta pre-trained model for general domains;

[0037] The pre-training framework language model is used to pre-train 3 epochs on the text data to obtain and save the pre-training weights.

[0038] In the above-mentioned text readability evaluation device, the annotation data set in the annotation scoring module includes a first-level data set with annotation scores between 0 and 1, a second-level data set with annotation scores between 1 and 2, a third-level data set with annotation scores between 2 and 3, a fourth-level data set with annotation scores between 3 and 4, and a fifth-level data set with annotation scores between 4 and 5, wherein the annotation scores are obtained by evaluation and scoring by an evaluator.

[0039] In the above-mentioned text readability evaluation device, the Roberta reading difficulty scoring model in the model building module is built in the following way:

[0040] Constructing a Roberta-Base model, loading the pre-trained weights of the roberta model in the general field, and adding two fully connected layers and multiple parallel dropout layers to each layer;

[0041] Input the output data of each pre-trained layer into the first fully connected layer, and input the output data of the first fully connected layer into the plurality of dropout layers respectively;

[0042] The output data of multiple dropouts are summed and input into the second fully connected layer, wherein the second fully connected layer is a classification fully connected layer.

[0043] In the above-mentioned text readability evaluation device, the training and scoring module trains the initial Roberta reading difficulty scoring model, including:

[0044] During the training process, different learning rates are set for different pre-training layers. FGM adversarial training is added to perform perturbations on the Embedding layer to increase the robustness of the model. Dynamic padding is used to reduce the amount of model calculations.

[0045] In the above-mentioned text readability evaluation device, when the training scoring module trains the initial Roberta reading difficulty scoring model, the loss function of each layer is the sum of the first loss function and the second loss function, where:

[0046] The first loss function is the CrossEntropy loss function:

[0047]

[0048] Where x is the input labeled data set, class is the category to be calculated, and y is the overall category;

[0049] The second loss function is a loss function based on contrastive learning:

[0050]

[0051] Where i represents the positive sample of the labeled data set, j represents the negative sample of the labeled data set, and λ i,j represents the contrast loss of the labeled dataset, z i is the vector representation of data i, z j is the vector representation of data j, z k is the vector representation of the corresponding data k in the negative sample of the labeled data set, N is the number of samples in the labeled data set, and sim() represents calculating the similarity between the positive sample and the negative sample of the labeled data set.

[0052] In the above-mentioned text readability evaluation device, the training scoring module adds and averages the output classification probabilities of all attention layers during the process of predictive scoring evaluation to obtain the final output probability, among which the one with the largest probability is used as the output score value.

[0053] In a third aspect, an embodiment of the present invention provides an article reading difficulty scoring device, comprising at least one control processor and a memory for communicating with the at least one control processor; the memory stores computer executable instructions, and the computer executable instructions are executed by the at least one control processor to enable the at least one control processor to execute the text readability assessment method described in the embodiment of the first aspect above.

[0054] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the text readability evaluation method as described in the embodiment of the first aspect above.

[0055] The embodiment of the present invention includes: a text readability evaluation method, a text readability evaluation device and a computer-readable storage medium. The text readability evaluation method includes the following steps: obtaining text data of reading materials; pre-training the roberta model of a general field according to the text data to obtain pre-training weights; selecting part of the data from the text data for annotation and scoring to construct an annotation data set; building an initial Roberta reading difficulty scoring model, loading the pre-training weights of the roberta model of the general field after pre-training as initial weights; training the initial Roberta reading difficulty scoring model to obtain a trained Roberta reading difficulty scoring model; inputting the text data of the reading difficulty to be judged into the trained Roberta reading difficulty scoring model for predictive scoring evaluation. The technical solution of this embodiment can help administrators, students, literacy course developers, and teachers who select texts to quickly and accurately evaluate the reading difficulty of articles, which can effectively help them choose reading texts, and can help students benefit from feedback on the complexity and readability of reading works, thereby making it easier to improve basic reading skills.

[0056] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings are used to provide a further understanding of the technical solution of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the technical solution of the present invention and do not constitute a limitation on the technical solution of the present invention.

[0058] The present invention is further described below in conjunction with the accompanying drawings and embodiments;

[0059] Figure 1 is a flow chart of a text readability evaluation method provided by an embodiment of the present invention;

[0060] Figure 2 is a flow chart of a text readability evaluation method provided by another embodiment of the present invention;

[0061] Figure 3is a schematic diagram of a Roberta reading difficulty scoring model provided by an embodiment of the present invention;

[0062] Figure 4 is a schematic diagram of a multi-layer dropout module provided by an embodiment of the present invention;

[0063] Figure 5 is a structural schematic diagram of a text readability evaluation device provided by an embodiment of the present invention;

[0064] Figure 6 The present invention is a schematic diagram of a device for scoring the difficulty of reading an article provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0065] This section will describe in detail the specific embodiments of the present invention. The preferred embodiments of the present invention are shown in the accompanying drawings. The purpose of the accompanying drawings is to supplement the description of the text part of the specification with graphics, so that people can intuitively and vividly understand each technical feature and the overall technical solution of the present invention, but it cannot be understood as a limitation on the scope of protection of the present invention.

[0066] In the description of the present invention, if there is a description of first and second, it is only for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0067] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0068] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0069] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0070] The embodiments of the present invention provide a text readability evaluation method, a text readability evaluation device, and a computer-readable storage medium, which can help administrators, students, literacy course developers, and teachers who select texts to quickly and accurately evaluate the reading difficulty of an article.

[0071] The embodiments of the present invention are further described below in conjunction with the accompanying drawings.

[0072] Reference Figure 1 , Figure 1 1 is a flowchart of a text readability evaluation method provided by an embodiment of the first aspect of the present invention. The article reading difficulty scoring method includes but is not limited to step S110, step S120, step S130, step S140, step S150 and step S160, wherein:

[0073] Step S110: obtaining text data of reading materials;

[0074] Step S120: pre-training a general-domain roberta model according to the text data to obtain pre-training weights; wherein the general-domain roberta model may be a general-purpose roberta model trained based on databases such as Chinese Wikipedia, Baidu Encyclopedia, Sina and Weibo;

[0075] Step S130: Selecting part of the text data for annotation and scoring to construct an annotation data set;

[0076] Step S140: constructing an initial Roberta reading difficulty scoring model, loading the pre-trained weights of the general domain Roberta model as initial weights;

[0077] Step S150: using the annotated data set to train the initial Roberta reading difficulty scoring model to obtain a trained Roberta reading difficulty scoring model;

[0078] Step S160: inputting the text data of the to-be-determined reading difficulty into the trained Roberta reading difficulty scoring model for predictive scoring evaluation.

[0079] It should be noted that the text data to be judged for reading difficulty are generally data sets such as novels, fairy tales, and stories.

[0080] Reference Figure 2 In the above-mentioned text readability evaluation method, the pre-training of the roberta model of the general field according to the text data in step S120 includes but is not limited to steps S210 and S220, wherein:

[0081] Step S210: Loading the roberta pre-trained model of the general field;

[0082] Step S220: using NVIDIA Nasdaq's continuous pre-training framework Megatron-LM language model, pre-training 3 epochs on the text data, obtaining pre-training weights and saving them.

[0083] In the above-mentioned text readability evaluation method, the annotated data set in step S130 includes a first-tier data set with annotated scores between 0 and 1, a second-tier data set with annotated scores between 1 and 2, a third-tier data set with annotated scores between 2 and 3, a fourth-tier data set with annotated scores between 3 and 4, and a fifth-tier data set with annotated scores between 4 and 5, wherein the annotation scores are obtained by evaluation and scoring by the evaluator.

[0084] It should be noted that, in some embodiments, when constructing an annotated data set, a portion of the text data of the to-be-determined reading difficulty obtained in step S110 is generally selected for manual annotation and scoring, and the scoring range is between 0 and 5 points, which are divided into 5 grades when annotated. For example, 0 to 1 point is the first grade, with the lowest reading difficulty; 4 to 5 points are the fifth grade, with the highest reading difficulty. Manual scoring adopts the method of taking the average of multiple scores, that is, four annotators are asked to score respectively, and the average of their scores is rounded off to obtain the final difficulty score. It can be understood that the data selected from the acquired text data of the to-be-determined reading difficulty for constructing the annotated data set is generally representative data.

[0085] In some other embodiments, due to the high professional requirements of Chinese reading difficulty assessment, university teachers majoring in Chinese and education can be hired to evaluate and score the selected data set. Each article requires three teachers to independently evaluate and score. The teachers directly give the score. The optional score decibels are 1 point, 2 points, 3 points, 4 points and 5 points, where 1 point indicates the lowest reading difficulty and 5 points indicates the highest reading difficulty. The scoring criterion is to take the mode of the evaluation results of the three teachers. If the three evaluation results are inconsistent, the results will be randomly sent to the fourth teacher for evaluation, and then the mode will be selected from the four scoring results.

[0086] Reference Figure 3 In the above-mentioned text readability evaluation method, the Roberta reading difficulty scoring model in step S140 is constructed in the following manner:

[0087] Constructing a Roberta-Base model, loading the pre-trained weights of the roberta model in the general field, and adding two fully connected layers and multiple parallel dropout layers to each layer;

[0088] Input the output data of each pre-trained layer into the first fully connected layer, and input the output data of the first fully connected layer into the plurality of dropout layers respectively;

[0089] The output data of multiple dropouts are summed and input into the second fully connected layer, wherein the second fully connected layer is a classification fully connected layer.

[0090] Specifically, refer to Figure 4 , the output data of each pre-trained layer is input into the first fully connected layer, and then transmitted as input data to the parallel multi-layer dropout module. The multi-layer dropout module includes four dropout layers. Finally, the outputs of the four dropout layers are summed as output and input into the classification fully connected layer.

[0091] It should be noted that the multi-layer dropout module Figure 4 As shown in the figure, different sentence representations are obtained through multiple parallel dropout layers, and then the corresponding positions of the obtained sentence representations are added to obtain the final output, and finally connected to the second fully connected layer for prediction. In this way, the annotation information is introduced into the model in advance to increase the generalization ability of the model. It is understandable that the other 12 layers are all modified according to this method to introduce prediction information in advance.

[0092] In the above-mentioned text readability evaluation method, the training of the initial Roberta reading difficulty scoring model in step S150 includes:

[0093] During the training process, different learning rates are set for different pre-training layers. FGM adversarial training is added to perform perturbations on the Embedding layer to increase the robustness of the model. Dynamic padding is used to reduce the amount of model calculations.

[0094] It is understandable that in order to speed up model convergence and cope with the situation after adding multiple fully connected layers, the model sets different learning rates for each multi-head attention layer and the external fully connected layer during training fine-tuning. Set a smaller learning rate in the shallower multi-head self-attention layer, a larger learning rate in the deeper multi-head attention layer, and the maximum learning rate in the outermost fully connected layer. For example, the numerical ratio corresponding to the learning rate divided into 4 parts is (multi-head attention 1-4 layers): (multi-head attention 5-8 layers): (multi-head attention 9-12 layers): (external fully connected layer) = (learn ing_rate / 1.6): (Learn ing_rate): (learn ing_rate*1.6): (learn ing_rate*10).

[0095] In addition, in order to cope with the characteristics of children's reading data, which is sometimes long and sometimes short, dynamic padding is used for the data during the training and fine-tuning stage. That is, the maximum padding length of the data is equal to the maximum length of the data in a batch and changes dynamically. This can avoid data loss caused by truncation and also avoid computational waste caused by excessive padding length.

[0096] In the above-mentioned text readability evaluation method, in the process of training the initial Roberta reading difficulty scoring model, the loss function of each layer is the sum of the first loss function and the second loss function, where:

[0097] The first loss function is the CrossEntropy loss function:

[0098]

[0099] Where x is the input labeled data set, class is the category to be calculated, and y is the overall category;

[0100] The second loss function is a loss function based on contrastive learning:

[0101]

[0102] Where i represents the positive sample of the labeled data set, j represents the negative sample of the labeled data set, and λ i,j represents the contrast loss of the labeled dataset, z i is the vector representation of data i, z j is the vector representation of data j, z k is the vector representation of the corresponding data k in the negative sample of the labeled data set, N is the number of samples in the labeled data set, and sim() represents calculating the similarity between the positive sample and the negative sample of the labeled data set.

[0103] In the above-mentioned text readability evaluation method, during the process of predictive scoring evaluation, the output classification probabilities of all attention layers are added and averaged to obtain the final output probability, among which the one with the largest probability is used as the output score value.

[0104] The technical solution of this embodiment can help administrators, students, literacy course developers, teachers who select texts, and other personnel to quickly and accurately assess the reading difficulty of articles, which can effectively help them select reading texts and can help students benefit from feedback on the complexity and readability of the reading works, thereby making it easier to improve basic reading skills.

[0105] Reference Figure 5The second aspect of the present invention provides a text readability evaluation device, including an acquisition module 510, a pre-training module 520, a marking and scoring module 530, a model building module 540 and a training and scoring module 550, wherein:

[0106] The acquisition module 510 is used to acquire text data of the reading material;

[0107] The pre-training module 520 is used to pre-train the Roberta model of the general field according to the text data to obtain the pre-training weights;

[0108] The annotation and scoring module 530 is used to select part of the data from the text data for annotation and scoring to construct an annotation data set;

[0109] The model building module 540 is used to build an initial Roberta reading difficulty scoring model, and load the pre-trained weights of the general field Roberta model as the initial weights;

[0110] The training and scoring module 550 is used to train the initial Roberta reading difficulty scoring model using the annotated data set to obtain a trained Roberta reading difficulty scoring model; and to input the text data of the to-be-determined reading difficulty into the trained Roberta reading difficulty scoring model for predictive scoring evaluation.

[0111] In the above-mentioned text readability evaluation device, the pre-training module 520 pre-trains the Roberta model of the general field according to the text data to obtain the pre-training weights, including:

[0112] Load the roberta pre-trained model for general domains;

[0113] The NVIDIA Nasdaq continuous pre-training framework Megatron-LM language model is used to pre-train 3 epochs on the text data to obtain and save the pre-training weights.

[0114] In the above-mentioned text readability evaluation device, the annotation data set in the annotation scoring module 530 includes a first-level data set with annotation scores between 0 and 1, a second-level data set with annotation scores between 1 and 2, a third-level data set with annotation scores between 2 and 3, a fourth-level data set with annotation scores between 3 and 4, and a fifth-level data set with annotation scores between 4 and 5, wherein the annotation scores are obtained by evaluation and scoring by an evaluator.

[0115] In the above-mentioned text readability evaluation device, the Roberta reading difficulty scoring model in the model building module 540 is built in the following manner:

[0116] Constructing a Roberta-Base model, loading the pre-trained weights of the roberta model in the general field, and adding two fully connected layers and multiple parallel dropout layers to each layer;

[0117] Input the output data of each pre-trained layer into the first fully connected layer, and input the output data of the first fully connected layer into the plurality of dropout layers respectively;

[0118] The output data of multiple dropouts are summed and input into the second fully connected layer, wherein the second fully connected layer is a classification fully connected layer.

[0119] In the above-mentioned text readability evaluation device, the training and scoring module 550 trains the initial Roberta reading difficulty scoring model, including:

[0120] During the training process, different learning rates are set for different pre-training layers. FGM adversarial training is added to perform perturbations on the Embedding layer to increase the robustness of the model. Dynamic padding is used to reduce the amount of model calculations.

[0121] In the above-mentioned text readability evaluation device, when the training scoring module 550 trains the initial Roberta reading difficulty scoring model, the loss function of each layer is the sum of the first loss function and the second loss function, where:

[0122] The first loss function is the CrossEntropy loss function:

[0123]

[0124] Where x is the input labeled data set, class is the category to be calculated, and y is the overall category;

[0125] The second loss function is a loss function based on contrastive learning:

[0126]

[0127] Where i represents the positive sample of the labeled data set, j represents the negative sample of the labeled data set, and λ i,j represents the contrast loss of the labeled dataset, z i is the vector representation of data i, z jis the vector representation of data j, z k is the vector representation of the corresponding data k in the negative sample of the labeled data set, N is the number of samples in the labeled data set, and sim() represents calculating the similarity between the positive sample and the negative sample of the labeled data set.

[0128] In the above-mentioned text readability evaluation device, the training scoring module 550 adds and averages the output classification probabilities of all attention layers during the process of predictive scoring evaluation to obtain the final output probability, among which the one with the largest probability is used as the output score value.

[0129] In addition, refer to Figure 6 The third aspect of the present invention provides an article reading difficulty scoring device 600, comprising at least one control processor 610 and a memory 620 for communicating with the at least one control processor 610; the memory 620 stores computer executable instructions, and the computer executable instructions are executed by the at least one control processor 610 to enable the at least one control processor 610 to execute the text readability evaluation method described in the first aspect of the present invention, for example, to execute Figure 1 Steps S110 to S160 of the method, or executing Figure 2 The method includes steps S210 to S220.

[0130] The control processor 610 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present disclosure;

[0131] The memory 620 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 620 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 620, and the control processor 610 is called to execute the text readability evaluation method of the embodiment of the present disclosure.

[0132] The article reading difficulty scoring device 600 provided in the embodiment of the present disclosure can be set in a terminal or in a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer or a smart watch, etc.; the server can be configured as an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0133] The disclosed embodiments may be used in a variety of general or special computer system environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices.

[0134] In addition, a fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, wherein the computer-executable instructions are used to enable a computer to execute the text readability evaluation method described in the first aspect of the present invention, such as executing Figure 1 Steps S110 to S160 of the method, or executing Figure 2 The method includes steps S210 to S220.

[0135] The embodiment of the present invention includes a text readability evaluation method, a text readability evaluation device and a computer-readable storage medium. Through reading the text data of the material; pre-training the Roberta model of the general field according to the text data to obtain the pre-training weight; selecting part of the data from the text data for annotation and scoring to construct an annotation data set; building an initial Roberta reading difficulty scoring model, loading the pre-training weights of the Roberta model of the general field after pre-training as the initial weights; training the initial Roberta reading difficulty scoring model to obtain a trained Roberta reading difficulty scoring model; inputting the text data of the reading difficulty to be judged into the trained Roberta reading difficulty scoring model for predictive scoring evaluation; can help administrators, students, literacy course developers and teachers who select texts to quickly and accurately evaluate the reading difficulty of articles, which can effectively help them choose reading texts, and can help students benefit from feedback on the complexity and readability of reading works, so as to improve basic reading skills more easily.

[0136] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed methods above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium or a non-transitory medium and a communication medium or a temporary medium. As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disk DVD or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0137] The embodiments of the present invention are described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Various changes can be made within the knowledge scope of ordinary technicians in the technical field without departing from the purpose of the present invention.

Claims

1. A method for evaluating text readability, characterized in that: include: Get the text data of the reading material; Pre-training a general domain roberta model according to the text data to obtain pre-training weights; Selecting part of the text data for annotation and scoring to construct an annotation data set; Building an initial Roberta reading difficulty scoring model, loading the general field Roberta model for pre-training and using the pre-trained weights as initial weights; Using the annotated data set to train the initial Roberta reading difficulty scoring model to obtain a trained Roberta reading difficulty scoring model; Inputting the text data to be judged for reading difficulty into the trained Roberta reading difficulty scoring model for predictive scoring evaluation; in: The pre-training of the roberta model in the general field according to the text data to obtain the pre-training weights includes: Load the roberta pre-trained model for general domains; The pre-training framework language model is used to pre-train 3 epochs on the text data to obtain and save the pre-training weights; The Roberta reading difficulty scoring model is constructed in the following way: Constructing a Roberta-Base model, loading the pre-trained weights of the roberta model in the general field, and adding two fully connected layers and multiple parallel dropout layers to each layer; Input the output data of each pre-trained layer into the first fully connected layer, and input the output data of the first fully connected layer into the plurality of dropout layers respectively; The output data of the plurality of dropouts are summed and input into the second fully connected layer, wherein the second fully connected layer is a classification fully connected layer; The training of the initial Roberta reading difficulty scoring model includes: During the training process, different learning rates are set for different pre-training layers. FGM adversarial training is added to perturb the Embedding layer to increase the robustness of the model. Dynamic padding is used to reduce the amount of model calculations.

2. The text readability evaluation method according to claim 1, characterized in that: The annotated data set includes a first-tier data set with annotated scores between 0 and 1, a second-tier data set with annotated scores between 1 and 2, a third-tier data set with annotated scores between 2 and 3, a fourth-tier data set with annotated scores between 3 and 4, and a fifth-tier data set with annotated scores between 4 and 5, wherein the annotated scores are obtained by evaluation and scoring by an evaluator.

3. The text readability evaluation method according to claim 1, characterized in that: In the process of training the initial Roberta reading difficulty scoring model, the loss function of each layer is the sum of the first loss function and the second loss function, where: The first loss function is the CrossEntropy loss function: Where x is the input labeled data set, class is the category to be calculated, and y is the overall category; The second loss function is a loss function based on contrastive learning: Where i represents the positive sample of the labeled data set, j represents the negative sample of the labeled data set, and λ i,j represents the contrast loss of the labeled dataset, z i is the vector representation of data i, z j is the vector representation of data j, z k is the vector representation of the corresponding data k in the negative sample of the labeled data set, N is the number of samples in the labeled data set, and sim() represents calculating the similarity between the positive sample and the negative sample of the labeled data set.

4. The text readability evaluation method according to claim 1, characterized in that: In the process of predictive scoring evaluation, the output classification probabilities of all attention layers are added and averaged to obtain the final output probability, and the one with the largest probability is used as the output score value.

5. A text readability evaluation device, characterized in that: include: An acquisition module is used to acquire text data of reading materials; A pre-training module, used to pre-train the Roberta model of a general field according to the text data to obtain pre-training weights; A marking and scoring module, used for selecting part of the text data for marking and scoring to construct a marked data set; A model building module, used to build an initial Roberta reading difficulty scoring model, and load the pre-trained weights of the Roberta model of the general field as initial weights; A training and scoring module, for training the initial Roberta reading difficulty scoring model using the annotated data set to obtain a trained Roberta reading difficulty scoring model; and for inputting text data to be judged for reading difficulty into the trained Roberta reading difficulty scoring model for predictive scoring evaluation; in: The pre-training of the roberta model in the general field according to the text data to obtain the pre-training weights includes: Load the roberta pre-trained model for general domains; The pre-training framework language model is used to pre-train 3 epochs on the text data to obtain and save the pre-training weights; The Roberta reading difficulty scoring model is constructed in the following way: Constructing a Roberta-Base model, loading the pre-trained weights of the roberta model in the general field, and adding two fully connected layers and multiple parallel dropout layers to each layer; Input the output data of each pre-trained layer into the first fully connected layer, and input the output data of the first fully connected layer into the plurality of dropout layers respectively; The output data of the plurality of dropouts are summed and input into the second fully connected layer, wherein the second fully connected layer is a classification fully connected layer; The training of the initial Roberta reading difficulty scoring model includes: During the training process, different learning rates are set for different pre-training layers. FGM adversarial training is added to perturb the Embedding layer to increase the robustness of the model. Dynamic padding is used to reduce the amount of model calculations.

6. A device for scoring the difficulty of reading articles, characterized in that: The invention comprises at least one control processor and a memory for communicating with the at least one control processor; the memory stores computer executable instructions, and the computer executable instructions are executed by the at least one control processor so that the at least one control processor can execute the text readability assessment method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the text readability evaluation method according to any one of claims 1 to 4.

Images