Text readability analysis methods, systems, and electronic devices based on NLP processing

Abstract

This application discloses a text readability analysis method, system, and electronic device based on NLP processing. The method extracts multiple character entities; performs word segmentation on the preprocessed text to be analyzed to obtain a word sequence and constructs a dependency syntax tree; based on the multiple character entities, the preprocessed text to be analyzed, and the dependency syntax tree, it extracts features such as the number of characters, the entropy of character personality diversity, the conflict density, the dialogue ratio, the relationship density, and the relationship complexity; it processes and fuses these features to obtain a fused feature vector; it inputs the fused feature vector into an encoder to obtain a preliminary score vector; and it determines the readability score of the text to be analyzed based on the preliminary score vector. This application can improve the accuracy of text readability analysis.

Description

Technical Field

[0001] This application relates to the field of text information processing technology, and in particular to a text readability analysis method, system and electronic device based on NLP processing. Background Technology

[0002] Text readability analysis is a core topic at the intersection of Natural Language Processing (NLP) and educational technology. Its goal is to assess the reading difficulty and comprehension efficiency of texts through quantitative indicators. Traditional readability assessment tools (such as Flesch-Kincaid and the SMOG index) mainly rely on surface language features (such as sentence length, syllable count, and lexical complexity). Although widely used in basic education and document simplification, they have significant limitations, mainly in the lack of semantic depth, neglect of narrative structure, character interaction, and emotional tension, and inability to distinguish between texts with "simple syntax but complex plots" and texts with "complex syntax but straightforward logic."

[0003] Therefore, existing technologies are not very accurate in analyzing text readability. Summary of the Invention

[0004] This application aims to propose a text readability analysis method, system, and electronic device based on NLP processing, which can improve the accuracy of text readability analysis.

[0005] In a first aspect, embodiments of this application provide a text readability analysis method based on NLP processing, the method comprising:

[0006] Data preprocessing is performed on the text to be analyzed to obtain the preprocessed text to be analyzed.

[0007] Extract multiple character entities from the preprocessed text to be analyzed;

[0008] The preprocessed text to be analyzed is segmented into words to obtain a word sequence, and a dependency syntax tree is constructed based on the word sequence.

[0009] Based on the multiple character entities, the preprocessed text to be analyzed, and the dependency syntax tree, the following features are extracted: character quantity feature, character personality diversity entropy feature, conflict density feature, dialogue proportion feature, relationship density feature, and relationship complexity feature. Specifically, the character quantity feature is the statistical number of characters in the preprocessed text to be analyzed; the character personality diversity entropy feature represents the diversity of character personalities; the conflict density feature represents the degree of conflict between words in the text to be analyzed; the dialogue proportion feature is the proportion of dialogue in the text to be analyzed; the relationship density feature represents the relationship density between two character entities in the text to be analyzed; and the relationship complexity feature represents the complexity of the relationships connected to character entities.

[0010] The number of characters, the entropy of character personality diversity, the conflict density, the dialogue ratio, the relationship density, and the relationship complexity are processed and fused to obtain a fused feature vector.

[0011] The fused feature vector is input into the encoder to obtain a preliminary scoring vector;

[0012] Based on the preliminary scoring vector, the readability score of the text to be analyzed is determined.

[0013] Compared with the prior art, the first aspect of this application has the following beneficial effects:

[0014] This method preprocesses the text to be analyzed to obtain the preprocessed text; extracts multiple character entities from the preprocessed text; segments the preprocessed text into word sequences, and constructs a dependency syntax tree based on these word sequences; extracts character quantity features, character personality diversity entropy features, conflict density features, dialogue ratio features, relation density features, and relation complexity features based on the multiple character entities, the preprocessed text, and the dependency syntax tree; processes and fuses these features to obtain a fused feature vector; inputs the fused feature vector into the encoder to obtain a preliminary score vector; and determines the readability score of the text to be analyzed based on the preliminary score vector. Thus, constructing a dependency syntax tree based on word sequences can better parse sentence grammatical structures, laying a solid data foundation for subsequent feature extraction. By comprehensively considering features such as the number of characters, the entropy of character personality diversity, the conflict density, the proportion of dialogue, the relationship density, and the relationship complexity, the accuracy of text readability analysis can be improved. Furthermore, some features are effectively correlated with readers' actual reading behavior, thereby enhancing the interpretability of text readability analysis.

[0015] In some implementations, the extraction of character quantity features, character personality diversity entropy features, conflict density features, dialogue proportion features, relationship density features, and relationship complexity features based on the multiple character entities, the preprocessed text to be analyzed, and the dependency syntax tree includes:

[0016] ;

[0017] in, This represents the entropy characteristic of character personality diversity. Indicates the first Character personality tags The proportion, This indicates a summation operation.

[0018] In some implementations, the extraction of character quantity features, character personality diversity entropy features, conflict density features, dialogue proportion features, relationship density features, and relationship complexity features based on the multiple character entities, the preprocessed text to be analyzed, and the dependency syntax tree includes:

[0019] Based on the multiple character entities and the preprocessed text to be analyzed, a character relationship graph is constructed using a character relationship extraction model.

[0020] Extract the path length relationship features between two character entities from the dependency syntax tree;

[0021] The path length relationship features and the preprocessed text to be analyzed are fused to obtain an enhanced feature vector;

[0022] The enhanced feature vector is input into the role relationship extraction model to extract relationship triples, and the number of extracted relationship triples is counted to obtain the total number of relationships.

[0023] The total number of characters in the preprocessed text to be analyzed is counted, and the relation density feature is determined based on the total number of characters and the total number of relations.

[0024] Based on the character relationship diagram, determine the number of relationships connected to each character node;

[0025] The number of characters is counted based on the multiple character entities, and the relationship complexity feature is determined based on the number of characters and the number of relationships connected to each character node.

[0026] In some implementations, the extraction of character quantity features, character personality diversity entropy features, conflict density features, dialogue proportion features, relationship density features, and relationship complexity features based on the multiple character entities, the preprocessed text to be analyzed, and the dependency syntax tree includes:

[0027] Interpret the speaker by analyzing related speakers;

[0028] The dialogue content of the speaker in the preprocessed text to be analyzed is located by identifying dialogue boundaries.

[0029] The number of words in the dialogue content is counted, and the total number of words in the preprocessed text to be analyzed is also counted.

[0030] The dialogue percentage feature is determined based on the number of words in the dialogue and the total number of words.

[0031] In some implementations, the extraction of character quantity features, character personality diversity entropy features, conflict density features, dialogue proportion features, relationship density features, and relationship complexity features based on the multiple character entities, the preprocessed text to be analyzed, and the dependency syntax tree includes:

[0032] ;

[0033] in, Indicates conflict density characteristics, This represents the total number of conflicts in the preprocessed text to be analyzed. Indicates the first The conflict type weight of each conflicting word. Indicates the first The emotional emphasis of the conflicting words, This indicates the total number of characters in the preprocessed text to be analyzed.

[0034] In some implementations, inputting the fused feature vector into the encoder to obtain a preliminary scoring vector includes:

[0035] The self-attention mechanism in the encoder captures the interaction relationships between features in the fused feature vector and assigns attention weights to features with interaction relationships.

[0036] Based on the assigned attention weights, the fused feature vectors are integrated through the feedforward network and layer normalization in the encoder to obtain a preliminary score vector.

[0037] In some implementations, determining the readability score of the text to be analyzed based on the preliminary scoring vector includes:

[0038] The initial scoring vector is input into the fully connected layer to obtain the output result of the fully connected layer;

[0039] The output of the fully connected layer is mapped through an activation function to obtain the readability score of the text to be analyzed.

[0040] Secondly, embodiments of this application also provide a text readability analysis system based on NLP processing, the system comprising:

[0041] The data processing unit is used to preprocess the text to be analyzed to obtain the preprocessed text to be analyzed.

[0042] An entity extraction unit is used to extract multiple character entities from the preprocessed text to be analyzed.

[0043] The syntax tree construction unit is used to perform word segmentation on the preprocessed text to be analyzed to obtain a word sequence, and to construct a dependency syntax tree based on the word sequence;

[0044] The feature extraction unit is used to extract the following features based on the multiple character entities, the preprocessed text to be analyzed, and the dependency syntax tree: character quantity feature, character personality diversity entropy feature, conflict density feature, dialogue proportion feature, relationship density feature, and relationship complexity feature. The character quantity feature is the statistical number of characters in the preprocessed text to be analyzed. The character personality diversity entropy feature is used to characterize the diversity of character personalities. The conflict density feature is used to characterize the degree of conflict between words in the text to be analyzed. The dialogue proportion feature is the proportion of dialogue in the text to be analyzed. The relationship density feature is used to characterize the relationship density between two character entities in the text to be analyzed. The relationship complexity feature is used to characterize the complexity of the relationships connected to character entities.

[0045] The feature fusion unit is used to perform feature processing and fusion on the character quantity feature, the character personality diversity entropy feature, the conflict density feature, the dialogue ratio feature, the relationship density feature, and the relationship complexity feature to obtain a fused feature vector.

[0046] The scoring vector acquisition unit is used to input the fused feature vector into the encoder to obtain a preliminary scoring vector;

[0047] A readability scoring unit is used to determine the readability score of the text to be analyzed based on the preliminary scoring vector.

[0048] Thirdly, embodiments of this application also provide an electronic device, including at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor, the instructions being executed by the at least one control processor to enable the at least one control processor to perform a text readability analysis method based on NLP processing as described above.

[0049] Fourthly, embodiments of this application also provide a computer-readable storage medium storing computer-executable instructions for causing a computer to perform a text readability analysis method based on NLP processing as described above.

[0050] It is understood that the beneficial effects of the second to fourth aspects compared with the related technologies are the same as the beneficial effects of the first aspect compared with the related technologies. Please refer to the relevant description in the first aspect above, which will not be repeated here. Attached Figure Description

[0051] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0052] Figure 1 This is a flowchart illustrating an embodiment of the text readability analysis method based on NLP processing provided in this application;

[0053] Figure 2 This is a schematic diagram of the overall architecture in the best embodiment of the NLP-based text readability analysis method provided in this application;

[0054] Figure 3 This is a schematic diagram of the data preprocessing and word segmentation layer structure in the best embodiment of the NLP-based text readability analysis method provided in this application;

[0055] Figure 4 This is a schematic diagram of the structure of an embodiment of the NLP-based text readability analysis system provided in this application;

[0056] Figure 5 This is a schematic diagram of the structure of an embodiment of the electronic device provided in this application. Detailed Implementation

[0057] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0058] In the description of this application, the use of terms such as "first," "second," etc., is for the purpose of distinguishing technical features only and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or the order of the technical features indicated.

[0059] In the description of this application, it should be understood that the orientation descriptions, such as up, down, etc., are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0060] In the description of this application, it should be noted that, unless otherwise explicitly defined, terms such as "setup," "installation," and "connection" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this application in conjunction with the specific content of the technical solution.

[0061] Traditional readability assessment tools (such as Flesch-Kincaid and the SMOG index) primarily rely on surface language features (e.g., sentence length, syllable count, and lexical complexity). While widely used in basic education and document simplification, they have significant limitations. These limitations mainly manifest in a lack of semantic depth, neglecting narrative structure, character interaction, and deeper logical elements such as emotional tension. They also fail to distinguish between texts with simple syntax but complex plots and texts with complex syntax but straightforward logic. Therefore, existing technologies for text readability analysis are not highly accurate.

[0062] To address the issue of low accuracy in existing text readability analysis technologies, this application proposes a text readability analysis method, system, and electronic device based on NLP processing.

[0063] Reference Figure 1 This application provides a flowchart illustrating a text readability analysis method based on NLP processing. This NLP-based text readability analysis method is applied to electronic devices, such as servers or mobile terminals. Figure 1 As shown, this NLP-based text readability analysis method may include the following steps:

[0064] Step S101: Perform data preprocessing on the text to be analyzed to obtain the preprocessed text to be analyzed;

[0065] Step S102: Extract multiple character entities from the preprocessed text to be analyzed;

[0066] Step S103: Perform word segmentation on the preprocessed text to be analyzed to obtain a word sequence, and construct a dependency syntax tree based on the word sequence;

[0067] Step S104: Based on multiple character entities, the preprocessed text to be analyzed, and the dependency syntax tree, extract the following features: number of characters, character personality diversity entropy, conflict density, dialogue proportion, relation density, and relation complexity. Among them, the number of characters is the number of characters in the preprocessed text to be analyzed; the character personality diversity entropy is used to represent the diversity of character personalities; the conflict density is used to represent the degree of conflict between words in the text to be analyzed; the dialogue proportion is the proportion of dialogue in the text to be analyzed; the relation density is used to represent the relation density between two character entities in the text to be analyzed; and the relation complexity is used to represent the complexity of the relationship connected to the character entity.

[0068] Step S105: Perform feature processing and fusion on the feature of the number of characters, the entropy of the diversity of character personalities, the feature of conflict density, the feature of dialogue ratio, the feature of relationship density, and the feature of relationship complexity to obtain the fused feature vector.

[0069] Step S106: Input the fused feature vector into the encoder to obtain the preliminary scoring vector;

[0070] Step S107: Determine the readability score of the text to be analyzed based on the preliminary scoring vector.

[0071] In this embodiment, the text to be analyzed is preprocessed to obtain the preprocessed text; multiple character entities are extracted from the preprocessed text; the preprocessed text is segmented to obtain a word sequence, and a dependency syntax tree is constructed based on the word sequence; based on the multiple character entities, the preprocessed text, and the dependency syntax tree, features such as the number of characters, the entropy of character personality diversity, the conflict density, the dialogue ratio, the relationship density, and the relationship complexity are extracted; these features are processed and fused to obtain a fused feature vector; the fused feature vector is input into the encoder to obtain a preliminary score vector; and the readability score of the text to be analyzed is determined based on the preliminary score vector. Thus, constructing a dependency syntax tree based on word sequences can better parse sentence grammatical structures, laying a solid data foundation for subsequent feature extraction. By comprehensively considering features such as the number of characters, the entropy of character personality diversity, the conflict density, the proportion of dialogue, the relationship density, and the relationship complexity, the accuracy of text readability analysis can be improved. Furthermore, some features are effectively correlated with readers' actual reading behavior, thereby enhancing the interpretability of text readability analysis.

[0072] The above-mentioned data preprocessing of the text to be analyzed can be based on regular expression processing to process the text to be analyzed into a plain text file.

[0073] The above-mentioned extraction of multiple character entities from the preprocessed text to be analyzed can be achieved by using entity recognition and coreference resolution to accurately locate the character entities in the text to be analyzed.

[0074] The above-mentioned preprocessed text to be analyzed is segmented into words. This can be done by using the existing LTP 4.0 sentence segmentation model to segment the preprocessed text into a sentence sequence, and then using the existing BERT-wwm model to segment the sentence sequence into a word sequence.

[0075] In some implementations, based on multiple character entities, preprocessed text to be analyzed, and dependency syntax trees, features such as the number of characters, the entropy of character personality diversity, the conflict density, the proportion of dialogue, the relation density, and the relation complexity are extracted, including:

[0076] ;

[0077] in, This represents the entropy characteristic of character personality diversity. Indicates the first Character personality tags The proportion, This indicates a summation operation.

[0078] In this embodiment, by analyzing the entropy of character personality diversity, we can know the complexity of the character personality in the text to be analyzed, which lays a good data foundation for the subsequent calculation of the text readability score, thereby improving the accuracy of the text readability score.

[0079] In some implementations, based on multiple character entities, preprocessed text to be analyzed, and dependency syntax trees, features such as the number of characters, the entropy of character personality diversity, the conflict density, the proportion of dialogue, the relation density, and the relation complexity are extracted, including:

[0080] Based on multiple character entities and preprocessed text to be analyzed, a character relationship graph is constructed using a character relationship extraction model.

[0081] Extract the path length relationship features between two character entities from the dependency syntax tree;

[0082] The path length relationship features are fused with the preprocessed text to be analyzed to obtain the enhanced feature vector;

[0083] The enhanced feature vector is input into the role relationship extraction model to extract relationship triples, and the number of extracted relationship triples is counted to obtain the total number of relationships.

[0084] The total number of characters in the preprocessed text to be analyzed is counted, and the relation density feature is determined based on the total number of characters and the total number of relations.

[0085] Based on the character relationship diagram, determine the number of relationships connected to each character node;

[0086] The number of characters is counted based on multiple character entities, and the relationship complexity feature is determined based on the number of characters and the number of relationships connected to each character node.

[0087] In this embodiment, by fusing the path length relationship features with the preprocessed text to be analyzed, and then extracting relationship triples based on the obtained enhanced feature vector, the accuracy of relationship triple extraction can be improved, thereby calculating accurate relationship density features. Furthermore, by determining the accurate relationship complexity features based on the number of characters and the number of relationships connected to each character node, a good data foundation can be laid for the subsequent calculation of the text readability score, thereby improving the accuracy of the text readability score.

[0088] The aforementioned role relationship extraction model can be a role relationship extraction model constructed using the TPLinker model.

[0089] The aforementioned extraction of path length relation features between two character entities from a dependency syntax tree can be achieved by extracting the number of dependency relations between the headword and dependency words or between dependency words in the dependency syntax tree, and then determining the path length relation features based on the number of these dependency relations. For example, if there are six dependency relations between the headword and dependency words (i.e., explicit relation triples) and five indirect dependency relations between dependency words (i.e., non-explicit association triples) between two character entities, then the number of corresponding dependency relations between the two character entities is 11. This number of corresponding dependency relations between the two character entities can then be used as the path length relation feature.

[0090] The above-mentioned method of fusing path length relationship features with preprocessed text to obtain enhanced feature vectors can be achieved by concatenation, addition, or fusion through attention mechanisms.

[0091] The above-mentioned inputting the enhanced feature vector into the role relationship extraction model to extract relationship triples can be done by inputting the enhanced feature vector into the role relationship extraction model to extract explicit relationship triples and non-explicit association triples.

[0092] In some implementations, based on multiple character entities, preprocessed text to be analyzed, and dependency syntax trees, features such as the number of characters, the entropy of character personality diversity, the conflict density, the proportion of dialogue, the relation density, and the relation complexity are extracted, including:

[0093] Interpret the speaker by analyzing related speakers;

[0094] Dialogue boundary identification is used to locate the speaker's dialogue content in the preprocessed text to be analyzed.

[0095] The number of words in the dialogue content is counted, and the total number of words in the preprocessed text to be analyzed is also counted.

[0096] The proportion of dialogue is determined based on the number of words in the dialogue and the total number of words.

[0097] In this embodiment, by determining the proportion of dialogue, it is possible to effectively correlate it with the reader's actual reading behavior, thereby laying a good data foundation for the subsequent calculation of the text's readability score and improving the accuracy of the text's readability score.

[0098] The speaker parsing and dialogue boundary recognition and localization mentioned above are techniques in natural language processing, which will not be described in detail in this embodiment.

[0099] In some implementations, based on multiple character entities, preprocessed text to be analyzed, and dependency syntax trees, features such as the number of characters, the entropy of character personality diversity, the conflict density, the proportion of dialogue, the relation density, and the relation complexity are extracted, including:

[0100] ;

[0101] in, Indicates conflict density characteristics, This represents the total number of conflicts in the preprocessed text to be analyzed. Indicates the first The conflict type weight of each conflicting word. Indicates the first The emotional emphasis of the conflicting words, This indicates the total number of characters in the preprocessed text to be analyzed.

[0102] In this embodiment, by calculating the conflict density feature, it is possible to better correlate it with the reader's actual reading behavior, thereby laying a good data foundation for the subsequent calculation of the text's readability score and improving the accuracy of the text's readability score.

[0103] In some implementations, the fused feature vector is input into the encoder to obtain a preliminary scoring vector, including:

[0104] The self-attention mechanism in the encoder is used to capture the interaction relationship between features in the fused feature vector, and attention weights are assigned to features with interaction relationships.

[0105] Based on the assigned attention weights, the fused feature vectors are integrated through the feedforward network and layer normalization in the encoder to obtain a preliminary score vector.

[0106] In this embodiment, by assigning attention weights to features with interactive relationships, the model can focus on key related features, thereby improving the accuracy of the text readability score.

[0107] The encoder mentioned above can be a Transformer encoder.

[0108] In some implementations, a readability score for the text to be analyzed is determined based on a preliminary scoring vector, including:

[0109] The initial scoring vector is input into the fully connected layer to obtain the output result of the fully connected layer;

[0110] The output of the fully connected layer is mapped through an activation function to obtain the readability score of the text to be analyzed.

[0111] In this embodiment, obtaining the readability score of the text to be analyzed through an accurate preliminary scoring vector can improve the accuracy of the text readability score.

[0112] To facilitate understanding by those skilled in the art, a set of preferred embodiments is provided below:

[0113] Text readability analysis is a core topic at the intersection of Natural Language Processing (NLP) and educational technology. Its goal is to assess the reading difficulty and comprehension efficiency of texts through quantitative indicators. Traditional readability assessment tools (such as Flesch-Kincaid and the SMOG index) mainly rely on surface language features (such as sentence length, syllable count, and lexical complexity). Although widely used in basic education and document simplification, they have significant limitations, mainly in the lack of semantic depth, neglect of narrative structure, character interaction, and emotional tension, and inability to distinguish between texts with "simple syntax but complex plots" and texts with "complex syntax but straightforward logic."

[0114] Existing technologies suffer from low accuracy and poor interpretability in text readability analysis. This is because most existing methods focus on a single dimension (e.g., analyzing only sentiment or syntax) and lack systematic integration of multimodal features. Furthermore, readability scores are not effectively correlated with readers' actual reading behavior, resulting in insufficient model interpretability.

[0115] To address the problems existing in the prior art, the technical solution of this embodiment includes the following:

[0116] Reference Figure 2 This embodiment adopts a modular pipeline architecture, divided into a data preprocessing and word segmentation layer, a parallel feature extraction layer, and a comprehensive evaluation layer, supporting high-concurrency processing and independent module expansion performance. Specifically, it includes:

[0117] (1) Data preprocessing and word segmentation layer.

[0118] Reference Figure 3 This step is a core foundational module of the Natural Language Processing (NLP) pipeline. Its goal is to transform raw unstructured text into standardized, semantically clear structured data, providing high-quality input for subsequent feature extraction.

[0119] The text cleaning stage involves using regular expressions to match and delete HTML tags and uncommon symbols, replacing full-width punctuation with half-width characters based on rules, and outputting a UTF-8 encoded plain text file with noise characters accounting for less than 0.1%, thus obtaining the preprocessed text (i.e. the text to be analyzed).

[0120] Sentence segmentation and tokenization are performed on the preprocessed text: continuous text is divided into sentences and lexical units. Specifically, a sentence segmentation model based on LTP 4.0 (F1=0.94) can handle complex sentences containing ellipses. Then, the BERT-wwm model is used to adaptively segment the segmented sentences to obtain word sequences.

[0121] Entity recognition and coreference resolution (NER and Coreference Resolution): These are used to identify named entities in text and resolve referential ambiguities. Specifically, a sequence labeling model based on SpanBERT can identify entity types, such as people, places, organizations, and times, but this embodiment only uses person / role entities. Coreference resolution uses SpanBERT trained on the OntoNotes dataset (a large-scale multilingual corpus) to improve the accuracy of resolving person / role aliases and solve long-distance referential problems. The Named Entity Recognition (NER) model SpanBERT can accurately locate person / role names, and combining it with a domain dictionary can further improve the recall rate.

[0122] Dependency parsing is used to parse the grammatical structure of sentences, extract core components such as subject, verb, and object, and represent them as a dependency parsing tree. Its main process is as follows: 1. Word segmentation and part-of-speech tagging: The sentence is segmented into a sequence of words (using the sentence segmentation and word segmentation methods described above). Each word in the sequence is tagged with its part of speech (noun, verb, adjective, etc.); 2. Dependency relation identification: Determines whether dependency relations exist between all word pairs in the sentence, and what type of dependency relations exist. The final output is a set of dependency relation triples (including dependency words, relation types, and headwords). Commonly used dependency parsing methods include rule-based methods, statistical methods, or neural network-based methods. This paper uses a neural network-based method, utilizing the Transformer's self-attention mechanism to automatically learn distributed representations (word vectors) of words and part-of-speech tags, and predicts dependency relations based on these rich distributed representations.

[0123] (2) Parallel feature extraction layer.

[0124] The core function of the character information extraction module is to perform personality analysis on the characters in the text (i.e., the pre-processed text to be analyzed). It presets 45 character personalities, classifies the character personalities using a Large Language Model (LLM), and calculates the proportion of each personality type. .

[0125] The formula for calculating the entropy of personality diversity for all characters is:

[0126] ;

[0127] in, Indicates the first Character personality tags The proportion, This represents summation, which calculates the entropy of all character personalities to obtain the entropy of character personality diversity.

[0128] The role relationship extraction module (i.e., the TPLinker model) aims to construct an interaction network (i.e., a role relationship graph) between character entities, quantifying the type and strength of dependency relationships. Based on the TPLinker model, this module achieves end-to-end joint relationship extraction, simultaneously identifying entity pairs and their relationship types, thus solving the error accumulation problem of traditional Pipeline methods.

[0129] Relation density is defined as the number of relations per unit text length:

[0130] ;

[0131] in, The total number of relations (obtained by using the TPLinker model to count explicit relation triples and non-explicit association triples, resulting in the total number of relation triples). ), This represents the total number of words in the text.

[0132] Specifically, using the Transformer-based dependency parser described earlier, a dependency parsing tree for the word sequence is constructed. Each word has its headword, dependency relation type, and dependent words. The tree structure of this dependency parsing tree represents the grammatical relationships between words in the sentence.

[0133] The dependency syntax tree is transformed into a feature that can be utilized by the relation joint extraction model (i.e., the TPLinker model). The path length relationship feature between two character entities (e1 and e2) in the dependency syntax tree is directly extracted as a discrete feature. That is, there are multiple dependency words and head words between the two character entities, and there is a path between the two connecting words. The number of paths between these words is counted, which is the path length relationship feature. The path length relationship feature is input into the relation joint extraction model to extract relation triples.

[0134] The original sentence text (i.e., the preprocessed text) and the generated dependency information representation (i.e., discrete features) are concatenated, added, or fused through an attention mechanism to obtain the enhanced input representation (i.e., the enhanced feature vector). This enhanced input representation is then fed into the Relation Joint Extraction (TPLinker) model. The TPLinker model marks the start and end positions of the character entities, as well as the start and end positions of the two character entities in a relation triplet. It predicts the relationship type between the character entity pairs (relation type refers to triples with explicit relational triples and triples with implicit associations), thus obtaining all extracted relation triples. The number of extracted relation triples is counted to obtain the total number of relations M.

[0135] Define the average degree of the network, which reflects the complexity of the relationships based on the average number of connections between nodes in the role relationship graph:

[0136] ;

[0137] in, For each character node The number of relationships connected to each character entity (based on the character node, directly count the number of all triples in the character relationship graph). The number of characters in the text (the total number of characters involved in the text extracted by LLM).

[0138] The dialogue detection module is responsible for locating dialogue content in the text and associating it with the speaker. Its technical implementation consists of two stages: dialogue boundary recognition and speaker parsing (both are techniques in natural language processing, which will not be described in detail in this embodiment). This module outputs a structured dialogue record (i.e., dialogue content), including the speaker, content text, and sentiment polarity (used for cross-validation with the sentiment module). It also calculates the dialogue proportion (i.e., dialogue word count / total text word count) and identifies densely packed dialogue segments.

[0139] The event conflict extraction module focuses on identifying adversarial events in text and assessing their narrative impact. This module employs a multi-level processing flow: First, conflict event boundaries are labeled using a predefined conflict trigger lexicon (including terms like "contest" and "betrayal") and the BERT-CRF model (i.e., conflict words are labeled). The conflict trigger lexicon includes conflict type weights (which are also user-defined); for example, larger conflicts such as "betrayal" and "war" have higher conflict type weights. Based on the labeled conflict event boundaries, the text is input into the Transformer model. After outputting the sentiment intensity of the conflict words, it is normalized using Softmax. Subsequently, semantic role labeling is used to determine the conflict subject and type (the purpose being to obtain conflict type weights). The output includes conflict density (number of times per thousand words), type distribution, and intensity curves, providing key tension indicators for readability analysis.

[0140] The conflict density formula defined in the event conflict quantification model is as follows:

[0141] ;

[0142] in, For conflict type weights, For emotional intensity, The total number of words in the text. This represents the total number of conflicts in the text (i.e., the number of conflicts equals the number of conflicting words).

[0143] (3) Scoring model layer (i.e., comprehensive evaluation layer).

[0144] As the decision-making center of the system, the scoring model layer is responsible for splicing and integrating the multi-dimensional features extracted by each module into a unified readable score. Its technical implementation covers feature fusion and dynamic weight allocation.

[0145] Feature input:

[0146] The input feature vector consists of six indicators obtained from four modules, including the number of characters, character personality diversity entropy, conflict density, dialogue ratio, relationship density, and relationship complexity. Z-score normalization is applied to indicators such as character diversity entropy and conflict density to eliminate dimensional differences. For long-tailed distributed indicators (such as the number of characters), Box-Cox transformation is used to improve normality.

[0147] Feature embedding: The normalized or Box-Cox transformed feature vectors (including the number of characters, the entropy of character personality diversity, the conflict density, the proportion of dialogue, the relationship density, and the relationship complexity) are concatenated to generate a structured feature vector, resulting in a fused feature vector.

[0148] The preprocessed data of the six indicators is converted into structured feature embedding vectors (i.e., fused feature vectors) that the model can process. Specifically, the value of each indicator is mapped to a fixed-dimensional vector through an embedding layer, ultimately forming a feature sequence of length 6 (each position corresponds to the embedding vector of one indicator).

[0149] Transformer Encoder: Feature Interaction Modeling and Preliminary Scoring. The Transformer encoder, as the core module, performs deep modeling of the feature sequence, specifically:

[0150] Self-attention mechanism: Through multi-head self-attention calculation, it captures the complex interaction relationships between 6 indicators (e.g., the correlation between the number of characters and relationship density, the synergistic effect between conflict density and dialogue ratio, the mutual influence between emotional entropy and character personality diversity entropy, etc.).

[0151] Dynamic weight allocation: Based on the correlation between features (such as "high conflict density + high dialogue ratio" having a more significant impact on readability), attention weights are automatically assigned to different feature interactions, allowing the model to focus on key related features.

[0152] Output preliminary score: Based on the assigned attention weights, the modeled feature sequence is integrated through the encoder's feedforward network and layer normalization, and a preliminary score vector with dimension 1 is output (reflecting the basic impact of feature interaction on readability).

[0153] Rating integration: final readability rating generation.

[0154] The initial score vector output by the Transformer encoder is input into the fully connected layer (i.e., the linear transformation layer), and the vector is mapped to the final text readability score (i.e., the readability score of the text to be analyzed) through the activation function (ReLU) (the numerical range is set to 0-10).

[0155] Compared with the prior art, the technical solution of this embodiment has the following advantages:

[0156] 1. The scoring model layer achieves high-precision and high-efficiency readability assessment by dynamically fusing structured and semantic features. Experiments show that the system model in this embodiment is robust in cross-genre scenarios, with a MAE as low as 2.9 points, and has strong interpretability (SHAP value analysis).

[0157] 2. By integrating four core modules—character information extraction, character relationship extraction, dialogue detection, and event conflict extraction—a multi-dimensional text evaluation system is constructed, ultimately generating a readability score. This framework not only focuses on traditional readability indicators (such as sentence length and lexical complexity) but also introduces deep semantic features such as narrative structure, emotional tension, and character interaction, making it applicable to diverse text types including novels, scripts, and news reports. The research aims to address the problem of existing readability evaluation tools neglecting the internal logic and emotional expression of texts, providing more comprehensive analytical support for fields such as education, publishing, and human-computer interaction.

[0158] Reference Figure 4 This application also provides a text readability analysis system based on NLP processing. The system includes a data processing unit 401, an entity extraction unit 402, a syntax tree construction unit 403, a feature extraction unit 404, a feature fusion unit 405, a score vector acquisition unit 406, and a readability scoring unit 407, wherein:

[0159] The data processing unit 401 is used to perform data preprocessing on the text to be analyzed, so as to obtain the preprocessed text to be analyzed.

[0160] The entity extraction unit 402 is used to extract multiple character entities from the preprocessed text to be analyzed.

[0161] The syntax tree construction unit 403 is used to perform word segmentation on the preprocessed text to be analyzed, obtain word sequences, and construct a dependency syntax tree based on the word sequences;

[0162] Feature extraction unit 404 is used to extract character quantity features, character personality diversity entropy features, conflict density features, dialogue proportion features, relation density features, and relation complexity features based on multiple character entities, preprocessed text to be analyzed, and dependency syntax tree. Among them, the character quantity feature is the number of characters in the preprocessed text to be analyzed; the character personality diversity entropy feature is used to characterize the diversity of character personalities; the conflict density feature is used to characterize the degree of conflict between words in the text to be analyzed; the dialogue proportion feature is the proportion of dialogue in the text to be analyzed; the relation density feature is used to characterize the relation density between two character entities in the text to be analyzed; and the relation complexity feature is used to characterize the complexity of the relationship connected to the character entity.

[0163] The feature fusion unit 405 is used to process and fuse the feature of the number of characters, the entropy of the character personality diversity, the conflict density, the dialogue ratio, the relationship density, and the relationship complexity to obtain the fused feature vector.

[0164] The scoring vector acquisition unit 406 is used to input the fused feature vector into the encoder to obtain the preliminary scoring vector;

[0165] The readability scoring unit 407 is used to determine the readability score of the text to be analyzed based on the preliminary scoring vector.

[0166] It should be noted that since the NLP-based text readability analysis system in this embodiment and the NLP-based text readability analysis method described above are based on the same inventive concept, the corresponding content in the method embodiment is also applicable to this system embodiment, and will not be described in detail here.

[0167] Reference Figure 5 This application also provides an electronic device, which includes:

[0168] At least one memory;

[0169] At least one processor;

[0170] At least one program;

[0171] The program is stored in memory, and the processor executes at least one program to implement the NLP-based text readability analysis method described above in this disclosure.

[0172] This electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0173] The electronic devices according to embodiments of this application will now be described in detail.

[0174] The processor 1600 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this disclosure.

[0175] The memory 1700 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1700 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1700 and is called and executed by the processor 1600 to execute the NLP-based text readability analysis method of the embodiments of this disclosure.

[0176] The input / output interface 1800 is used to implement information input and output.

[0177] The communication interface 1900 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0178] Bus 2000 transmits information between various components of the device (e.g., processor 1600, memory 1700, input / output interface 1800, and communication interface 1900);

[0179] The processor 1600, memory 1700, input / output interface 1800 and communication interface 1900 are connected to each other within the device via bus 2000.

[0180] This disclosure also provides a storage medium, which is a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the above-described NLP-based text readability analysis method.

[0181] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0182] The embodiments described in this disclosure are for the purpose of more clearly illustrating the technical solutions of this disclosure and do not constitute a limitation on the technical solutions provided by this disclosure. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by this disclosure are also applicable to similar technical problems.

[0183] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this disclosure, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0184] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0185] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0186] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0187] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0188] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0189] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0190] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0191] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. The embodiments of this application have been described in detail above with reference to the accompanying drawings, but this application is not limited to the above embodiments. Various changes can be made within the scope of knowledge possessed by those skilled in the art without departing from the spirit of this application.

[0192] The embodiments of this application have been described in detail above with reference to the accompanying drawings. However, this application is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of this application.

Claims

1. A text readability analysis method based on NLP processing, characterized in that, The method comprises: data preprocessing is performed on the text to be analyzed to obtain preprocessed text to be analyzed; extracting multiple character entities in the preprocessed text to be analyzed; performing word segmentation processing on the preprocessed text to be analyzed to obtain a word sequence, and constructing a dependency syntax tree according to the word sequence; based on the multiple character entities, the preprocessed text to be analyzed and the dependency syntax tree, extracting a character number feature, a character personality diversity entropy feature, a conflict density feature, a dialogue proportion feature, a relationship density feature and a relationship complexity feature, wherein the character number feature is the number of character entities in the preprocessed text to be analyzed, the character personality diversity entropy feature is used to represent the diversity of character personalities, the conflict density feature is used to represent the degree of conflict of words in the text to be analyzed, the dialogue proportion feature is the proportion of dialogue in the text to be analyzed, the relationship density feature is used to represent the relationship density between two character entities in the text to be analyzed, and the relationship complexity feature is used to represent the complexity of the relationship connected with the character entity; performing feature processing and fusion on the character number feature, the character personality diversity entropy feature, the conflict density feature, the dialogue proportion feature, the relationship density feature and the relationship complexity feature to obtain a fused feature vector; inputting the fused feature vector into an encoder to obtain a preliminary score vector; determining the readability score of the text to be analyzed according to the preliminary score vector. 2.The NLP processing based text readability analysis method of claim 1, wherein, The method comprises: ； wherein, represents the character personality diversity entropy feature, represents the proportion of the class character personality label , represents the summation operation. 3.The NLP processing based text readability analysis method of claim 1, wherein, The method comprises: According to the multiple character entities and the preprocessed text to be analyzed, a character relationship extraction model is used to construct a character relationship graph; extracting a path length relationship feature between two character entities from the dependency syntax tree; fusing the path length relationship feature and the preprocessed text to be analyzed to obtain an enhanced feature vector; inputting the enhanced feature vector into the character relationship extraction model to extract relationship triples, and counting the number of extracted relationship triples to obtain a total relationship number; counting the total number of words in the preprocessed text to be analyzed, and determining the relationship density feature based on the total number of words and the total relationship number; determining the number of relationships connected with each character node according to the character relationship graph; counting the number of character entities according to the multiple character entities, and determining the relationship complexity feature based on the number of character entities and the number of relationships connected with each character node. 4.The NLP processing based text readability analysis method of claim 1, wherein, The role quantity feature, the role character diversity entropy feature, the conflict density feature, the dialogue proportion feature, the relationship density feature, and the relationship complexity feature are extracted based on the multiple character entity, the preprocessed text to be analyzed, and the dependency syntax tree, including: The speaker is parsed to associate the speaker; The dialogue content of the speaker in the preprocessed text to be analyzed is located by dialogue boundary identification; The dialogue word number of the dialogue content is counted, and the total word number of the preprocessed text to be analyzed is counted; The dialogue proportion feature is determined according to the dialogue word number and the total word number.

5. The NLP processing based text readability analysis method of claim 1, wherein, The role quantity feature, the role character diversity entropy feature, the conflict density feature, the dialogue proportion feature, the relationship density feature, and the relationship complexity feature are extracted based on the multiple character entity, the preprocessed text to be analyzed, and the dependency syntax tree, including: ； wherein, denotes the conflict density feature, denotes the total number of conflicts in the pre-processed text to be analyzed, denotes the conflict type weight of the th conflict word, denotes the sentiment emphasis of the th conflict word, denotes the total number of words of the pre-processed text to be analyzed.

6. The NLP processing based text readability analysis method of claim 1, wherein, The preliminary score vector is obtained by inputting the fused feature vector into the encoder, including: The interaction relationship between the features in the fused feature vector is captured by the self-attention mechanism in the encoder, and the features with the interaction relationship are assigned attention weights; According to the assigned attention weights, the fused feature vector is integrated by the feedforward network and the layer normalization in the encoder to obtain the preliminary score vector.

7. The NLP processing based text readability analysis method of claim 1, wherein, The readability score of the text to be analyzed is determined according to the preliminary score vector, including: The preliminary score vector is input into the full connection layer to obtain the full connection layer output result; The full connection layer output result is mapped by the activation function to obtain the readability score of the text to be analyzed.

8. A text readability analysis system based on NLP processing, characterized by, The system includes: A data processing unit configured to perform data preprocessing on a text to be analyzed to obtain a preprocessed text to be analyzed; An entity extraction unit configured to extract multiple character entities in the preprocessed text to be analyzed; A syntax tree construction unit configured to perform word segmentation processing on the preprocessed text to be analyzed to obtain a word sequence, and construct a dependency syntax tree according to the word sequence; A feature extraction unit configured to extract a role quantity feature, a role character diversity entropy feature, a conflict density feature, a dialogue proportion feature, a relationship density feature, and a relationship complexity feature based on the multiple character entities, the preprocessed text to be analyzed, and the dependency syntax tree, wherein the role quantity feature is the number of character roles in the preprocessed text to be analyzed, the role character diversity entropy feature is used to represent the diversity of character roles, the conflict density feature is used to represent the degree of conflict of words in the text to be analyzed, the dialogue proportion feature is the proportion of dialogue in the text to be analyzed, the relationship density feature is used to represent the relationship density between two character entities in the text to be analyzed, and the relationship complexity feature is used to represent the complexity of the relationship connected with the character entity. a feature fusion unit configured to perform feature processing and fusion on the character number feature, the character personality diversity entropy feature, the conflict density feature, the dialogue proportion feature, the relationship density feature, and the relationship complexity feature to obtain a fused feature vector; a score vector obtaining unit configured to input the fused feature vector into an encoder to obtain a preliminary score vector; a readability score unit configured to determine a readability score of the text to be analyzed according to the preliminary score vector.

9. An electronic device, comprising: The computer readable storage medium stores computer executable instructions for causing a computer to perform the NLP processing based text readability analysis method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer executable instructions for causing a computer to perform the NLP processing based text readability analysis method according to any one of claims 1 to 7.

Images