Chinese text readability evaluation method based on information gain and hierarchical classification

Abstract

The invention belongs to the technical field of natural language processing, and discloses a Chinese text readability evaluation method based on information gain and hierarchical classification, a readability evaluation model of the method comprises a language feature representation module, an information gain guided depth representation module and a hierarchical classification module; wherein the language feature representation module is used for extracting language features and mapping the features to a deep semantic space by adopting a natural language description mode to obtain corresponding language feature representation; the information gain guided depth representation module is used for fully mining the depth representation of the text by fusing the representations of a plurality of coding layers according to the difference of information contained in different coding layers by utilizing an information gain principle; the hierarchical classification module performs modeling to form a hierarchical tree structure, and learns the mutual relation between different readability difficulty categories by capturing hierarchical structure information. According to the method, the representation of a plurality of coding layers is fused to fully mine the depth representation of the text, and the mutual relation between different readability difficulty categories can be learned.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing, and in particular relates to a Chinese text readability evaluation method based on information gain and hierarchical classification. Background Art

[0002] Text readability refers to the ease with which readers can understand and absorb the text content. Automatic Readability Assessment (ARA) is a quantitative and automatic assessment and analysis of text difficulty, which plays a vital role in many fields such as education, publishing, information retrieval and natural language processing.

[0003] Early readability research established a direct connection between readability and language features by using readability formulas. Later, researchers used machine learning methods to automatically learn the relationship between language features and readability. However, evaluation methods based on language features can only simply utilize shallow features of the text and cannot capture more valuable semantic information, resulting in low evaluation accuracy. In contrast, pre-trained language models represented by BERT, with their powerful semantic modeling capabilities, have greatly surpassed previous model methods in multiple natural language processing tasks, and have also demonstrated great advantages in readability evaluation tasks. However, deep models also have some problems, such as poor interpretability and often neglecting the effective use of text language features.

[0004] In recent years, research on automatic readability assessment has gradually shifted to a method that combines manually extracted language features with deep features extracted by deep neural networks, namely the Hybrid Automatic Readability Assessment (HARA). However, most current models still have the following problems: First, when extracting text features, current pre-trained language models focus on semantic features, while ignoring features at other levels such as vocabulary and syntax, and fail to fully tap the deep features of the article. Second, most current models regard the text readability assessment task as a classification task with independent categories, ignoring the sequential relationship between difficulty levels, resulting in the difficulty difference relationship between different categories not being fully reflected. Summary of the invention

[0005] The purpose of the present invention is to provide a Chinese text readability evaluation method based on information gain and hierarchical classification, which is used to solve the technical problems that the prior art fails to fully explore the deep features of the article and ignores the hierarchical sequence relationship of the readability difficulty level, resulting in the difficulty difference relationship between different categories cannot be fully reflected.

[0006] The Chinese text readability assessment method based on information gain and hierarchical classification comprises: the method establishes a readability assessment model based on information gain and hierarchical classification, including a language feature representation module, a deep representation module guided by information gain and a hierarchical classification module; wherein the language feature representation module is used to extract language features and map these features to a deep semantic space in a natural language description manner to obtain corresponding language feature representations; the deep representation module guided by information gain uses a coding output layer containing semantic information as an anchor layer, and utilizes the information gain principle to fuse the representations of multiple coding layers according to the differences in information contained in different coding layers to fully mine the deep representation of the text; the hierarchical classification module is modeled as a hierarchical tree structure, and by capturing hierarchical structure information, the relationship between different readability difficulty categories is learned.

[0007] Preferably, in the information gain-guided deep representation module, the text is first divided into several paragraph blocks, which are then input into the pre-trained model together to generate respective encoding representations, and the encoding representations of these paragraph blocks are fused using the inter-block residual Transformer module to obtain the feature representations of all paragraph blocks in the encoding layer; the deep representation module selects to perform a separate classification evaluation task on the last encoding layer and calculates the separate information entropy of this layer; the deep representations of the previous encoding layers are sequentially spliced ​​with the deep representation of the last encoding layer, and the classification evaluation tasks are performed on each of them, and the conditional entropy obtained by fusion of each encoding layer with the last encoding layer is calculated, and the information entropy and conditional entropy are used as the loss function of the feature extraction loss; the method allows all gain coefficients to form a gain coefficient vector, and the adaptive weights of each layer to form a weight vector; the relationship between the gain coefficient vector and the weight vector is calculated KL Divergence is used as the guided loss, and the information gain guided deep representation module is trained jointly by the feature extraction loss and the guided loss.

[0008] Preferably, i Feature representation of all paragraph blocks in the layer encoding layer X i , which is expressed as follows: X i = IsR-Transformer ([ s i1 , s i2 ,⋯, s in ]), where for n paragraph blocks s 1 , s 2 ,⋯, s n , each encoding layer of the pre-trained model can generate its own encoding representation for these n paragraph blocks s i1, s i2 ,⋯, s in , IsR-Transformer The inter-block residual Transformer module fuses the encoded representations of these paragraph blocks; the information entropy is expressed as follows: , in, P i Indicates that it is classified as i The probability of the class, C Indicates the number of categories, y i represents the true label, H(X L ) Indicates L The information entropy of the layer coding layer.

[0009] The conditional entropy is expressed as follows: , in, P i Indicates L After concatenation of the layer coding layers, the classification evaluation is performed and is classified as i The probability of the class, C Indicates the number of categories, y i represents the true label, H ( X L │ X i ) indicates the i The coding layer and L The conditional entropy obtained by fusion of layer coding layers.

[0010] The loss function of the deep representation module is expressed as follows: , in, represents the loss for feature extraction on deep representation.

[0011] Preferably, the information gain generated by each layer is calculated: IG ( X L , X i )= H ( X L )- H ( X L │ X i ),in, IG ( X L , X i ) indicates the L The information gain generated by the fusion of the feature information of the i-th coding layer by the i-th coding layer; the calculated gain coefficient is used to represent the weight of guiding the fusion of the i-th coding layer of the pre-trained language model, which is expressed as: , in, IG ( X L , X j ) indicates the L Layer coding layer fusion j The information gain generated after the feature information of the layer encoding layer is fused j Layer encoding layer from the previous L -1 encoding layer to choose from, t represents the temperature coefficient, represents the first i The weights when layer encoding layers are fused.

[0012] forward L When the feature representation of the -1 layer is fused, weighted fusion is used to obtain the fused representation, and then this fused representation is combined with the L The depth representation of the layer is concatenated and the depth representation is calculated by linear transformation. The specific representation is as follows: , , in, X i For the i The feature representation of the layer encoding layer, β i For the i The weights of the feature representation of the layer encoding layer are learnable adaptive weights; X fused For the front L The fused representation obtained by weighted fusion of the depth representation of -1 layer, [ X L , X fused ] is the fusion representation and L The depth of the layer stitching, W and b are the weight matrix and offset of the linear transformation respectively, f deep is the final depth representation.

[0013] Preferably, let all gain coefficients Composition gain coefficient vector α , and the adaptive weights of each layer β i Composition weight vector β ; Calculate the gain coefficient vector α With the weight vector β Between KL Divergence is used as the guided loss, and the calculation process is expressed as follows: , in, represents the guidance loss, Represents the calculation gain coefficient vector α With the weight vector β Between KL Divergence.

[0014] The total loss function of the information gain-guided deep representation module can be expressed as follows: , in, To guide the loss, is the feature extraction loss, The depth guided by information gain represents the total loss of the module.

[0015] Preferably, the obtained language feature representation is projected into the deep feature space by means of an orthogonal projection vector, and the new language feature representation obtained thereby is as follows: , in, f' Ling For new language features, f Ling is the language feature representation output by the language feature representation module; finally, the new language feature representation f' Ling and deep feature representation f deep Splice them together to get the final text representation f all .

[0016] Preferably, in the hierarchical classification module, the sequence relationship based on the readability difficulty category is modeled as a hierarchical tree, and each level is used as a difficulty perception block. In the difficulty perception block, the method realizes the perception of difficulty through classification operations. Through the operation of multiple difficulty perception blocks, the difficulty perception granularity can be refined from high-level nodes to low-level nodes in sequence; when a classification error occurs, different penalties are given according to the distance between the error result and the target difficulty level, thereby reflecting the difficulty relationship between different categories. A hierarchical constraint penalty function is designed as a loss function to measure the impact of the node on its subsequent nodes if it is misclassified. In the case of high-level coarse-grained classification, if the prediction is wrong, it will be punished more severely.

[0017] Preferably, the difficulty perception block includes a feature transformation block, a classifier and a residual fusion gate. The method first represents the text f all Input to the feature transformation block for feature transformation operation, which is expressed as follows: , in, s 1 is the Sigmoid activation function, s 2 is the GELU activation function, W 1 and b 1 represents the weight matrix and offset of the first linear transformation in the feature transformation block, W 2 and b 2 Represents the weight matrix and offset of the second linear transformation in the feature transformation block, which is the transformed text representation.

[0018] In order to give the model a preliminary sense of difficulty, the classifier performs the first coarse-grained classification operation, which is expressed as follows: P i = classifier i ( f all ), P i For coarse-grained i The probability distribution of readability difficulty levels, classifier i Indicates the first i readability difficulty level classification; in order to alleviate the gradient disappearance, the hierarchical classification module uses a residual fusion gate to enhance the feature representation, which is specifically expressed as follows: , , in,[ f' all , f all ] represents the result of concatenating the transformed text representation with the previous text representation. W and b is the weight matrix and offset of the corresponding linear transformation, s 1 is the Sigmoid activation function, G represents the new feature obtained after the concatenation of text representations is linearly transformed and activated; ⊙ represents the dot product operation, f'' all Represents the new features and the transformed text f' all And the previous text representation f all The fused result is the enhanced feature output by the residual fusion gate.

[0019] Preferably, the hierarchical constraint penalty function is expressed as follows: , in, n is the tree height of the hierarchical tree, , represents the impact factor, where N i Indicates the current layer i The number of all descendant nodes of a node, n is the total number of corresponding nodes, p i represents the predicted probability value, y i represents the true label.

[0020] Preferably, the final loss function of this method is expressed as: , in, Loss is the total loss of the model in this method, l is a hyperparameter that balances the losses of both the information gain-guided deep representation module and the hierarchical classification module.

[0021] The present invention has the following advantages: This method uses the information gain-guided deep representation module to use the encoding output layer containing semantic information as the anchor layer, and uses the information gain principle to fuse the representations of multiple encoding layers according to the differences in the information contained in different encoding layers to fully mine the deep representation of the text. The hierarchical classification module is modeled as a hierarchical tree structure, and by capturing the hierarchical structure information, the relationship between different readability difficulty categories can be learned.

[0022] This method uses an inter-block residual Transformer module to fuse these paragraph block features, enrich the semantic representation coding layer, and eliminate the uncertainty of information, so as to represent the information gain of this information. In order to make the adaptive weights gradually converge to the actual importance of the features of each coding layer, this method uses the KL divergence (information divergence) between the gain coefficient vector and the weight vector as the guided loss. On the other hand, in order to enable these coding layers to more fully extract deep information, the deep representation module uses the information entropy and conditional entropy calculated above as the loss function to train the further feature extraction capabilities of each coding layer.

[0023] In the difficulty perception block, this method realizes the perception of difficulty through classification operations. Since the difficulty difference in high-level nodes is the largest when modeled as a hierarchical tree, it is easier for the model to distinguish the difficulty relationship between them. Through the operation of multiple difficulty perception blocks, the difficulty perception granularity can be refined from high-level nodes to low-level nodes in sequence. Therefore, when performing the final evaluation, this model can obtain the difficulty level of the text more accurately.

[0024] The language feature representation module adopted in this method extracts language features and maps these features to the deep semantic space using natural language description to solve the semantic space misalignment problem that may exist between traditional numerical language features and deep features. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a flow chart of the Chinese text readability evaluation method based on information gain and hierarchical classification of the present invention.

[0026] Figure 2 A schematic diagram of a hierarchical tree formed by modeling the hierarchical classification module in the present invention. DETAILED DESCRIPTION

[0027] The specific implementation modes of the present invention are further explained in detail below by describing the embodiments with reference to the accompanying drawings, so as to help those skilled in the art to have a more complete, accurate and in-depth understanding of the inventive concept and technical solution of the present invention.

[0028] like Figure 1-Figure 2As shown, the present invention provides a Chinese text readability evaluation method based on information gain and hierarchical classification, including: the method establishes a new readability evaluation model, which is a readability evaluation model based on information gain and hierarchical classification (IGHC-ARA, IG represents information gain, HC represents hierarchical classification). The new readability evaluation model (IGHC-ARA) includes a language feature representation module, an information gain-guided deep representation module and a hierarchical classification module. Among them, the language feature representation module is used to extract language features and map these features to a deep semantic space in a natural language description to solve the semantic space misalignment problem that may exist between traditional numerical language features and deep features. The information gain-guided deep representation module uses the encoding output layer containing semantic information as the anchor layer, and uses the information gain principle to fuse the representations of multiple encoding layers according to the differences in the information contained in different encoding layers to fully mine the deep representation of the text. The hierarchical classification module is modeled as a hierarchical tree structure, and the relationship between different readability difficulty categories is learned by capturing the hierarchical structure information.

[0029] The language feature representation module includes: a language feature extractor and a language feature interpreter. The language feature extractor is used to extract corresponding language features from the text, and the language feature interpreter uses natural language description to map the extracted language features to the deep semantic space to solve the semantic space misalignment problem that may exist between traditional numerical language features and deep features.

[0030] For the language feature representation module, this method first selects a series of representative language features as input, which include complex characters and words. The specific language features are shown in Table 1.

[0031] Table 1: Language features used in the language feature interpreter.

[0032] This method sets the language features in the text to include language features l 1 , l 2 ,⋯, l m , m represents the total number of language features. This method obtains the corresponding language features based on statistical methods, and uses "the number of [feature name] in the text is [value]" as the interpretation template of the language features. All language features are wrapped with the interpretation template, and the results are recorded as T 1 , T 2 ,⋯, Tm The result is input into the language feature interpreter to obtain the corresponding language feature representation. The calculation formula is as follows: f Ling = Interpreter ( T 1 ,T 2 ,⋯,T m ),in Interpreter Indicates that the language features in the brackets are processed using the language feature interpreter. f Ling Represents the language feature representation output by this module.

[0033] Information gain-guided deep representation module: According to the differences in the encoded information of different encoding layers of the pre-trained language model, the deep features at different levels are captured, and finally the deep features at different levels are fused using the information gain principle.

[0034] For the information gain guided deep representation module, this method first divides the text into n paragraph blocks s 1 , s 2 ,⋯, s n , and then input them into the pre-trained model together. Each encoding layer of the pre-trained model can generate its own encoding representation for these n paragraph blocks s i1 , s i2 ,⋯, s in ,in i =1,⋯, L , L is the number of coding layers. The depth representations obtained by different coding layers contain feature representations with different meanings. In order to better exchange information between each paragraph block, this method uses an Inter-section R-Transformer (IsR-Transformer) module to fuse the coded representations of these paragraph blocks to obtain i Feature representation of all paragraph blocks in the layer encoding layer X i , which is expressed as follows: X i = IsR-Transformer ([ s i1 , s i2 ,⋯, s in]), In order to enrich the encoding layer of semantic representation, the deep representation module calculates the first L Layer coding layers are fused sequentially before L The result of -1 coding layer eliminates the uncertainty of information, thereby indicating the degree of information gain of this information.

[0035] Since the last encoding layer of the pre-trained model contains the richest semantic information, the deep representation module selects L The feature representation of the layer encoding layer performs a separate classification evaluation task and calculates the separate information entropy of the layer. The information entropy is expressed as follows: , in, P i Indicates that it is classified as i The probability of the class, C Indicates the number of categories, y i represents the true label, H(X L ) Indicates L The information entropy of the layer coding layer.

[0036] Afterwards, in order to analyze L -How can the information in the first coding layer be used to L The encoding layer achieves the most effective complementarity, and the depth representation module converts the previous L The depth of the -1 coding layer is represented in sequence with the L The depth representation of each coding layer is concatenated, and each performs classification evaluation tasks, and calculates the difference between each coding layer and the first L The conditional entropy obtained by fusion of layer coding layers is expressed as follows: , in, P i Indicates L After concatenation of the layer coding layers, the classification evaluation is performed and is classified as i The probability of the class, C Indicates the number of categories, y i represents the true label, H ( X L │ X i ) indicates the i The coding layer and L The conditional entropy obtained by fusion of layer coding layers.

[0037] In order to enable these coding layers to more fully extract depth information, the depth representation module uses the information entropy and conditional entropy calculated above as the loss function to train the further feature extraction capabilities of each coding layer. The loss function is expressed as follows: , in, represents the loss for feature extraction on deep representation.

[0038] At the same time, this method uses the L The information entropy generated by the classification evaluation of the encoding layer is subtracted from the information entropy generated by the fusion layer. i The conditional entropy obtained after the layer encoding layer feature information is used to obtain the information gain generated by each layer: IG ( X L , X i )= H ( X L )- H ( X L │ X i ), in, IG ( X L , X i ) indicates the L Layer coding layer fusion i The information gain generated after the layer encoding layer feature information.

[0039] In order to highlight the important coding layer features, this method calculates the gain coefficient, which is used to represent the first i The weight of the layer coding layer fusion is expressed as: , in, IG ( X L , X j ) indicates the L Layer coding layer fusion j The information gain generated after the feature information of the layer encoding layer is fused j Layer encoding layer from the previous L -1 encoding layer to choose from, t represents the temperature coefficient, represents the first i The weights when layer encoding layers are fused.

[0040] Before the actual L When the feature representation of the -1 layer is fused, this method first uses a common weighted fusion to obtain the fused representation, and then combines this fused representation with the L The depth representation of the layer is concatenated and the depth representation is calculated by linear transformation. The specific representation is as follows: , , in, X i For the i The feature representation of the layer encoding layer, β i For the i The weights of the feature representation of the layer encoding layer are learnable adaptive weights; X fused For the front L The fused representation obtained by weighted fusion of the depth representation of -1 layer, [ X L , X fused ] is the fusion representation and L The depth of the layer stitching, W and b are the weight matrix and offset of the linear transformation respectively, f deep is the final depth representation.

[0041] Finally, in order to make the adaptive weights gradually converge to the actual importance of the features of each coding layer, this method makes all gain coefficients Composition gain coefficient vector α , and the adaptive weights of each layer β i Composition weight vector β ; Calculate the gain coefficient vector α With the weight vector β Between KL Divergence (information divergence) is used as the guidance loss, and the calculation process is expressed as follows: , in, represents the guidance loss, Represents the calculation gain coefficient vector α With the weight vector β Between KL Divergence.

[0042] This module is trained jointly by feature extraction loss and guidance loss, so the total loss function of the information gain guided deep representation module can be expressed as follows: , in, To guide the loss, is the feature extraction loss, The depth guided by information gain represents the total loss of the module.

[0043] Fusion of language feature representation and deep representation: The next step is to fuse the language feature representation and deep representation to obtain the final text representation. The specific method of this step is to project the obtained language feature representation into the deep feature space using an orthogonal projection vector to remove redundant information. The new language feature representation obtained is as follows: , in, f' Ling Finally, the new language feature is represented by f' Ling and deep feature representation f deep Splice them together to get the final text representation f all .

[0044] Hierarchical classification module: It is used to implement hierarchical text classification tasks, thereby capturing hierarchical structure information, so that the model can better learn the relationship between categories. The hierarchical classification module models the sequence relationship of readability difficulty categories as a hierarchical tree to better capture the difficulty differences between different categories, such as Figure 2 shown.

[0045] For the hierarchical tree, this method regards each level as a difficulty aware block (DAB), and the specific structure includes a feature transform block (FTB), a classifier and a residual fusion gate. Through the difficulty aware block, the perception of difficulties of different granularities can be gradually realized.

[0046] In the difficulty perception block, in order to obtain better feature representation, this method first converts the text representation f all Input to the feature transformation block (FTB) for feature transformation operation, which is expressed as follows: , in, s 1 is the Sigmoid activation function, s 2 is the GELU activation function, W 1 and b 1represents the weight matrix and offset of the first linear transformation in the feature transformation block, W 2 and b 2 The weight matrix and offset of the second linear transformation in the feature transformation block are the text representation after the transformation. In order to give the model a preliminary sense of difficulty, the classifier performs the first coarse-grained classification operation, which is expressed as follows: P i = classifier i ( f all ), P i For coarse-grained i The probability distribution of readability difficulty levels. classifier i Indicates the first i To alleviate the gradient vanishing, the hierarchical classification module uses a residual fusion gate to enhance the feature representation, which is specifically expressed as follows: , , in,[ f' all , f all ] represents the result of concatenating the transformed text representation with the previous text representation. W and b is the weight matrix and offset of the corresponding linear transformation, s 1 is the Sigmoid activation function, G represents the new feature obtained after the concatenation of text representations is linearly transformed and activated; ⊙ represents the dot product operation, f'' all Represents the new features and the transformed text f' all And the previous text representation f all The fused result is the enhanced feature output by the residual fusion gate.

[0047] In the difficulty perception block, this method realizes the perception of difficulty through classification operations. Since the difficulty difference in high-level nodes is the largest when modeled as a hierarchical tree, it is easier for the model to distinguish the difficulty relationship between them. Through the operation of multiple difficulty perception blocks, the difficulty perception granularity can be refined from high-level nodes to low-level nodes in sequence. Therefore, when performing the final evaluation, this model can obtain the difficulty level of the text more accurately.

[0048] In order to distinguish the "closeness" of prediction errors, that is, when a classification error occurs, different penalties are given according to the distance between the error result and the target difficulty level, thereby reflecting the difficulty relationship between different categories. A hierarchical constraint penalty function is also designed as the loss function in the hierarchical classification module, which is expressed as follows: , in, n is the tree height of the hierarchical tree, , represents the impact factor, where N i Indicates the current layer i The number of all descendant nodes of a node, n is the total number of corresponding nodes, p i represents the predicted probability value, y i Represents the true label. The hierarchical classification module uses a hierarchical constraint penalty function to measure the impact of a node misclassified on its subsequent nodes. If the model makes a wrong prediction in a high-level coarse-grained classification, it will be punished more severely.

[0049] During model training, this method uses the information gain to guide the loss of the depth representation module. and the loss of the hierarchical classification module Combined together, as the total loss function to train the entire model, the final loss function is expressed as: , in, Loss is the total loss of the model in this method, l is a hyperparameter that balances the losses of both the information gain-guided deep representation module and the hierarchical classification module.

[0050] The present invention is described above by way of example in conjunction with the accompanying drawings. It is obvious that the specific implementation of the present invention is not limited to the above-mentioned method. As long as various non-substantial improvements are made using the inventive concept and technical solution of the present invention, or the inventive concept and technical solution are directly applied to other occasions without improvement, they are all within the protection scope of the present invention.

Claims

1. A Chinese text readability evaluation method based on information gain and hierarchical classification, characterized by: include: This method establishes a readability evaluation model based on information gain and hierarchical classification, including a language feature representation module, an information gain-guided deep representation module and a hierarchical classification module; wherein, the language feature representation module is used to extract language features and map these features to a deep semantic space in a natural language description manner to obtain corresponding language feature representations; the information gain-guided deep representation module uses the encoding output layer containing semantic information as the anchor layer, and uses the information gain principle to fuse the representations of multiple encoding layers according to the differences in information contained in different encoding layers to fully mine the deep representation of the text; the hierarchical classification module is modeled as a hierarchical tree structure, and by capturing the hierarchical structure information, the relationship between different readability difficulty categories is learned.

2. The Chinese text readability evaluation method based on information gain and hierarchical classification according to claim 1 is characterized by: In the deep representation module guided by information gain, the text is first divided into several paragraph blocks, which are then input into the pre-trained model together to generate their own encoding representations. The encoding representations of these paragraph blocks are fused using the inter-block residual Transformer module to obtain the feature representations of all paragraph blocks in the encoding layer; the deep representation module selects the last encoding layer to perform a separate classification evaluation task and calculates the information entropy of the layer; the deep representations of the previous encoding layers are sequentially concatenated with the deep representation of the last encoding layer, and the classification evaluation tasks are performed on each layer, and the conditional entropy obtained by fusion of each encoding layer and the last encoding layer is calculated, and the information entropy and conditional entropy are used as the loss function of the feature extraction loss; this method allows all gain coefficients to form a gain coefficient vector, and the adaptive weights of each layer to form a weight vector; Calculate the relationship between the gain coefficient vector and the weight vector KL Divergence is used as the guided loss, and the information gain guided deep representation module is trained jointly by the feature extraction loss and the guided loss.

3. The Chinese text readability evaluation method based on information gain and hierarchical classification according to claim 2 is characterized by: No. i The feature representation of all paragraph blocks in the layer encoding layer X i , which is expressed as follows: X i = IsR-Transformer ([ s i1 , s i2 ,⋯, s in ]), where for n paragraph blocks s 1, s 2,⋯, s n , each encoding layer of the pre-trained model can generate its own encoding representation for these n paragraph blocks s i1 , s i2 ,⋯, s in , IsR-Transformer The inter-block residual Transformer module fuses the encoded representations of these paragraph blocks; the information entropy is expressed as follows: ， in, P i Indicates that it is classified as i The probability of the class, C Indicates the number of categories, y i represents the true label, H(X L ) Indicates L Information entropy of the layer coding layer; The conditional entropy is expressed as follows: ， in, P i Indicates L After concatenation of the layer coding layers, the classification evaluation is performed and is classified as i The probability of the class, C Indicates the number of categories, y i represents the true label, H ( X L │ X i ) indicates the i The coding layer and L The conditional entropy obtained by fusion of layer coding layers; The loss function of the deep representation module is expressed as follows: ， in, represents the loss for feature extraction on deep representation.

4. The Chinese text readability evaluation method based on information gain and hierarchical classification according to claim 3 is characterized by: Calculate the information gain generated by each layer: IG ( X L , X i )= H ( X L )- H ( X L │ X i ),in, IG ( X L , X i ) indicates the L The information gain generated by the fusion of the feature information of the i-th coding layer by the i-th coding layer; the calculated gain coefficient is used to represent the weight of guiding the fusion of the i-th coding layer of the pre-trained language model, which is expressed as: ， in, IG ( X L , X j ) indicates the L Layer coding layer fusion j The information gain generated after the feature information of the layer encoding layer is fused j Layer encoding layer from the previous L -1 encoding layer to choose from, τ represents the temperature coefficient, represents the first i The weights when layer coding layers are fused; forward L When the feature representation of the -1 layer is fused, weighted fusion is used to obtain the fused representation, and then this fused representation is combined with the L The depth representation of the layer is concatenated and the depth representation is calculated by linear transformation. The specific representation is as follows: ， ， in, X i For the i The feature representation of the layer encoding layer, β i For the i The weights of the feature representation of the layer encoding layer are learnable adaptive weights; X fused For the front L The fused representation obtained by weighted fusion of the depth representation of -1 layer, [ X L , X fused ] is the fusion representation and L The depth of the layer stitching, W and b are the weight matrix and offset of the linear transformation respectively, f deep is the final depth representation.

5. The Chinese text readability evaluation method based on information gain and hierarchical classification according to claim 4 is characterized by: Let all gain coefficients Composition gain coefficient vector α , and the adaptive weights of each layer β i Composition weight vector β ; Calculate the gain coefficient vector α With the weight vector β Between KL Divergence is used as the guided loss, and the calculation process is expressed as follows: ， in, represents the guidance loss, Represents the calculation gain coefficient vector α With the weight vector β Between KL Divergence; The total loss function of the information gain-guided deep representation module can be expressed as follows: ， in, To guide the loss, is the feature extraction loss, The depth guided by information gain represents the total loss of the module.

6. The Chinese text readability evaluation method based on information gain and hierarchical classification according to claim 5 is characterized by: The obtained language feature representation is projected into the deep feature space by using an orthogonal projection vector, and the new language feature representation obtained is as follows: ， in, f' Ling For new language features, f Ling is the language feature representation output by the language feature representation module; finally, the new language feature representation f' Ling and deep feature representation f deep Splice them together to get the final text representation f all .

7. The Chinese text readability evaluation method based on information gain and hierarchical classification according to claim 6 is characterized by: In the hierarchical classification module, the sequence relationship based on readability difficulty categories is modeled as a hierarchical tree, and each level is regarded as a difficulty perception block. In the difficulty perception block, this method realizes the perception of difficulty through classification operations. Through the operation of multiple difficulty perception blocks, the difficulty perception granularity can be refined from high-level nodes to low-level nodes in sequence; when a classification error occurs, different penalties are given according to the distance between the error result and the target difficulty level, thereby reflecting the difficulty relationship between different categories. A hierarchical constraint penalty function is designed as a loss function to measure the impact of a node misclassified on its subsequent nodes. If the prediction is wrong in the case of high-level coarse-grained classification, it will be punished more severely.

8. The Chinese text readability evaluation method based on information gain and hierarchical classification according to claim 7 is characterized by: The difficulty perception block includes a feature transformation block, a classifier, and a residual fusion gate. This method first represents the text f all Input to the feature transformation block for feature transformation operation, which is expressed as follows: ， in, σ 1 is the Sigmoid activation function, σ 2 is the GELU activation function, W 1 and b 1 represents the weight matrix and offset of the first linear transformation in the feature transformation block, W 2 and b 2 represents the weight matrix and offset of the second linear transformation in the feature transformation block, which is the text representation after the transformation; In order to give the model a preliminary sense of difficulty, the classifier performs the first coarse-grained classification operation, which is expressed as follows: P i = classifier i ( f all ), P i For coarse-grained i The probability distribution of readability difficulty levels, classifier i Indicates the first i readability difficulty level classification; in order to alleviate the gradient disappearance, the hierarchical classification module uses a residual fusion gate to enhance the feature representation, which is specifically expressed as follows: ， ， in,[ f' all , f all ] represents the result of concatenating the transformed text representation with the previous text representation. W and b is the weight matrix and offset of the corresponding linear transformation, σ 1 is the Sigmoid activation function, G represents the new feature obtained after the concatenation of text representations is linearly transformed and activated; ⊙ represents the dot product operation, f'' all Represents the new features and the transformed text f' all And the previous text representation f all The fused result is the enhanced feature output by the residual fusion gate.

9. The Chinese text readability evaluation method based on information gain and hierarchical classification according to claim 8 is characterized by: The hierarchical constraint penalty function is expressed as follows: ， in, n is the tree height of the hierarchical tree, , represents the impact factor, where N i Indicates the current layer i The number of all descendant nodes of a node, n is the total number of corresponding nodes, p i represents the predicted probability value, y i represents the true label.

10. The Chinese text readability evaluation method based on information gain and hierarchical classification according to claim 9 is characterized in that: The final loss function of this method is expressed as: ， in, Loss is the total loss of the model in this method, λ is a hyperparameter that balances the losses of both the information gain-guided deep representation module and the hierarchical classification module.

Images