Text readability analysis method and system based on NLP processing and electronic equipment

Abstract

The invention discloses a text readability analysis method and system based on NLP processing and electronic equipment. The method comprises the following steps: extracting a plurality of character role entities; performing word segmentation processing on the preprocessed to-be-analyzed text to obtain a word sequence, and constructing a dependency syntax tree; on the basis of the character role entities, the preprocessed to-be-analyzed text and the dependency syntax tree, character number features, character character diversity entropy features, conflict density features, dialogue proportion features, relation density features and relation complexity features are extracted; performing feature processing and fusion on the role number feature, the role character diversity entropy feature, the conflict density feature, the dialogue proportion feature, the relationship density feature and the relationship complexity feature to obtain a fused feature vector; inputting the fused feature vector into an encoder to obtain a preliminary score vector; and determining a readability score of the to-be-analyzed text according to the preliminary score vector. According to the method, the accuracy of text readability analysis can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of text information processing, in particular to a text readability analysis method and system based on NLP processing and an electronic device. BACKGROUND

[0002] Text readability analysis is a core topic in the intersection of natural language processing (NLP) and educational technology, aiming to evaluate the reading difficulty and understanding efficiency of a text through quantitative indicators. Traditional readability assessment tools (such as Flesch-Kincaid and SMOG index) mainly rely on surface language features (such as sentence length, syllable number, and vocabulary complexity), although they are widely used in basic education and document simplification, they have significant limitations, mainly in the absence of semantic depth, ignoring narrative structure, character interaction, and emotional tension, and cannot distinguish between texts with simple syntax but complex plots and texts with complex syntax but straightforward logic.

[0003] Therefore, the accuracy of the existing related technology for text readability analysis is not high. SUMMARY

[0004] The present application aims to provide a text readability analysis method and system based on NLP processing and an electronic device, which can improve the accuracy of text readability analysis.

[0005] In a first aspect, an embodiment of the present application provides a text readability analysis method based on NLP processing, the method comprising: performing data preprocessing on the text to be analyzed to obtain preprocessed text to be analyzed; extracting multiple character role entities in the preprocessed text to be analyzed; performing word segmentation processing on the preprocessed text to be analyzed to obtain a word sequence, and constructing a dependency syntax tree according to the word sequence; extracting role quantity features, role personality diversity entropy features, conflict density features, dialogue proportion features, relationship density features, and relationship complexity features based on the multiple character role entities, the preprocessed text to be analyzed, and the dependency syntax tree, wherein the role quantity features are the number of character roles in the preprocessed text to be analyzed, the role personality diversity entropy features are used to represent the diversity of character personalities, the conflict density features are used to represent the degree of conflict between words in the text to be analyzed, the dialogue proportion features are the proportion of dialogue in the text to be analyzed, the relationship density features are used to represent the relationship density between two character role entities in the text to be analyzed, and the relationship complexity features are used to represent the complexity of the relationship connected to the character role entity; Processing and fusing the character quantity feature, the character personality diversity entropy feature, the conflict density feature, the conversation ratio feature, the relationship density feature, and the relationship complexity feature to obtain a fused feature vector; Inputting the fused feature vector into an encoder to obtain a preliminary score vector; The readability score of the text to be analyzed is determined according to the preliminary score vector.

[0006] Compared with the prior art, the first aspect of the present application has the following beneficial effects: The method performs data preprocessing on a text to be analyzed to obtain a preprocessed text to be analyzed; extracts multiple character role entities from the preprocessed text to be analyzed; performs word segmentation on the preprocessed text to be analyzed to obtain a word sequence, and constructs a dependency syntax tree based on the word sequence; extracts character quantity features, character personality diversity entropy features, conflict density features, dialogue ratio features, relationship density features, and relationship complexity features based on the multiple character role entities, the preprocessed text to be analyzed, and the dependency syntax tree; performs feature processing and fusion on the character quantity features, character personality diversity entropy features, conflict density features, dialogue ratio features, relationship density features, and relationship complexity features to obtain a fused feature vector; inputs the fused feature vector into an encoder to obtain a preliminary score vector; and determines the readability score of the text to be analyzed based on the preliminary score vector. In this way, constructing a dependency syntax tree based on word sequences can better parse the grammatical structure of sentences and lay a good data foundation for the subsequent feature extraction; by comprehensively considering the number of characters, the entropy characteristics of character personality diversity, the conflict density characteristics, the dialogue ratio characteristics, the relationship density characteristics and the relationship complexity characteristics, the accuracy of text readability analysis can be improved, and some features are effectively associated with the reader's actual reading behavior, thereby improving the interpretability of text readability analysis.

[0007] In some embodiments, extracting a character quantity feature, a character personality diversity entropy feature, a conflict density feature, a dialogue ratio feature, a relationship density feature, and a relationship complexity feature based on the multiple character role entities, the preprocessed text to be analyzed, and the dependency syntax tree includes: ; in, Represents the character diversity entropy characteristics, Indicates the Character personality tags The proportion of Represents a sum operation.

[0008] In some embodiments, the role quantity feature, the role character diversity entropy feature, the conflict density feature, the dialogue proportion feature, the relationship density feature, and the relationship complexity feature are extracted based on the plurality of character entity, the preprocessed text to be analyzed, and the dependency syntax tree, including: According to the plurality of character entity and the preprocessed text to be analyzed, a character relationship extraction model is used to construct a character relationship graph; The path length relationship feature between two character entities is extracted from the dependency syntax tree; The path length relationship feature and the preprocessed text to be analyzed are fused to obtain an enhanced feature vector; The enhanced feature vector is input into the character relationship extraction model, relationship triplets are extracted, and the number of extracted relationship triplets is counted to obtain a total relationship number; The total number of words in the preprocessed text to be analyzed is counted, and the relationship density feature is determined based on the total number of words and the total relationship number; According to the character relationship graph, the number of relationships connected to each character node is determined; The number of character roles is counted according to the plurality of character entities, and the relationship complexity feature is determined based on the number of character roles and the number of relationships connected to each character node.

[0009] In some embodiments, the role quantity feature, the role character diversity entropy feature, the conflict density feature, the dialogue proportion feature, the relationship density feature, and the relationship complexity feature are extracted based on the plurality of character entity, the preprocessed text to be analyzed, and the dependency syntax tree, including: The speaker is associated by speaker analysis; The dialogue content of the speaker in the preprocessed text to be analyzed is located by dialogue boundary identification; The dialogue word number of the dialogue content is counted, and the total word number of the preprocessed text to be analyzed is counted; The dialogue proportion feature is determined according to the dialogue word number and the total word number.

[0010] In some embodiments, the role quantity feature, the role character diversity entropy feature, the conflict density feature, the dialogue proportion feature, the relationship density feature, and the relationship complexity feature are extracted based on the plurality of character entity, the preprocessed text to be analyzed, and the dependency syntax tree, including: ; Wherein, The conflict density feature is represented by The total number of conflicts in the preprocessed text to be analyzed is represented by a conflict type weight indicating a first a sentiment emphasis indicating a first a total number of words of the preprocessed text to be analyzed.

[0011] In some embodiments, the inputting the fused feature vector into an encoder to obtain a preliminary score vector comprises: capturing, by a self-attention mechanism in the encoder, an interaction relationship between features in the fused feature vector, and assigning an attention weight to a feature having the interaction relationship; integrating, according to the assigned attention weight, the fused feature vector by a feedforward network and layer normalization in the encoder to obtain the preliminary score vector.

[0012] In some embodiments, the determining the readability score of the text to be analyzed according to the preliminary score vector comprises: inputting the preliminary score vector into a fully connected layer to obtain a fully connected layer output result; mapping the fully connected layer output result by an activation function to obtain the readability score of the text to be analyzed.

[0013] In a second aspect, the embodiments of the present application further provide a text readability analysis system based on NLP processing, which comprises: a data processing unit configured to perform data preprocessing on a text to be analyzed to obtain a preprocessed text to be analyzed; an entity extraction unit configured to extract a plurality of character role entities in the preprocessed text to be analyzed; a syntax tree construction unit configured to perform word segmentation processing on the preprocessed text to be analyzed to obtain a word sequence, and construct a dependency syntax tree according to the word sequence; a feature extraction unit configured to extract a role number feature, a role character diversity entropy feature, a conflict density feature, a dialogue proportion feature, a relationship density feature, and a relationship complexity feature based on the plurality of character role entities, the preprocessed text to be analyzed, and the dependency syntax tree, wherein the role number feature is a number of character roles in the preprocessed text to be analyzed, the role character diversity entropy feature is used to represent the diversity of character roles, the conflict density feature is used to represent the degree of conflict of words in the text to be analyzed, the dialogue proportion feature is a proportion of dialogue in the text to be analyzed, the relationship density feature is used to represent the relationship density between two character role entities in the text to be analyzed, and the relationship complexity feature is used to represent the complexity of relationships connected to the character role entities; ​​a feature fusion unit, configured to perform feature processing and fusion on the character quantity feature, the character personality diversity entropy feature, the conflict density feature, the dialogue proportion feature, the relationship density feature, and the relationship complexity feature, to obtain a fused feature vector; a score vector obtaining unit, configured to input the fused feature vector into an encoder to obtain a preliminary score vector; a readability score unit, configured to determine a readability score of the text to be analyzed according to the preliminary score vector.

[0014] In a third aspect, an electronic device is provided, including at least one control processor and a memory in communication connection with the at least one control processor; the memory stores instructions executable by the at least one control processor, and the instructions are executed by the at least one control processor to enable the at least one control processor to perform the NLP processing-based text readability analysis method as described above.

[0015] In a fourth aspect, a computer readable storage medium is provided, which stores computer executable instructions for causing a computer to perform the NLP processing-based text readability analysis method as described above.

[0016] It can be understood that the beneficial effects of the second aspect to the fourth aspect compared with the related art are the same as the beneficial effects of the first aspect compared with the related art, and reference can be made to the related description in the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0017] The above and / or additional aspects and advantages of the present application will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which: Figure 1 is a flowchart of an embodiment of the NLP processing-based text readability analysis method provided by the present application; Figure 2 is a schematic diagram of the overall architecture in the best embodiment of the NLP processing-based text readability analysis method provided by the present application; Figure 3 is a schematic diagram of the data preprocessing and word segmentation layer structure in the best embodiment of the NLP processing-based text readability analysis method provided by the present application; Figure 4 is a schematic diagram of the structure of an embodiment of the NLP processing-based text readability analysis system provided by the present application; Figure 5 is a schematic diagram of the structure of an embodiment of the electronic device provided by the present application. DETAILED DESCRIPTION

[0018] Embodiments of the present application are described below in detail with reference to the accompanying drawings, wherein the same or similar components or components having the same or similar functions are denoted by the same or similar reference numerals throughout the drawings. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application and cannot be understood as a limitation of the present application.

[0019] In the description of the present application, if the first, second, etc. are described, it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features or the order of the indicated technical features.

[0020] In the description of the present application, it should be understood that the orientation description, such as the orientation or position relationship indicated by up, down, etc. is based on the orientation or position relationship shown in the drawings, only for the purpose of facilitating the description of the present application and simplifying the description, and is not intended to indicate or imply that the indicated device or element must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the present application.

[0021] In the description of the present application, it should be noted that, unless otherwise explicitly limited, the words such as setting, installing, connecting, etc. should be broadly understood, and those skilled in the art can reasonably determine the specific meaning of the above words in the present application in combination with the specific content of the technical solution.

[0022] Traditional readability assessment tools (such as Flesch-Kincaid and SMOG index) mainly rely on surface language features (such as sentence length, number of phonemes, and vocabulary complexity, etc.), although they are widely used in basic education and document simplification, but have significant limitations, mainly in the absence of semantic depth, ignoring narrative structure, role interaction and emotional tension and other deep logic, and cannot distinguish between "simple syntax but complex plot" texts and "complex syntax but straightforward logic" texts. Therefore, the accuracy of the text readability analysis of the prior art is not high.

[0023] To solve the problem of low accuracy of text readability analysis in the prior art, the present application provides a text readability analysis method and system based on NLP processing and an electronic device.

[0024] Reference Figure 1 The text readability analysis method based on NLP processing provided by the embodiments of the present application is shown in the flowchart. The text readability analysis method based on NLP processing is applied to an electronic device, which can be a server or a mobile terminal, etc. As shown in Figure 1 The text readability analysis method based on NLP processing can include the following steps: Step S101, data preprocessing is performed on the text to be analyzed to obtain a preprocessed text to be analyzed; Step S102, multiple character entity in the preprocessed text to be analyzed is extracted; Step S103, the preprocessed text to be analyzed is subjected to word segmentation processing to obtain a word sequence, and a dependency syntax tree is constructed according to the word sequence; Step S104, based on the multiple character entity, the preprocessed text to be analyzed and the dependency syntax tree, a character number feature, a character personality diversity entropy feature, a conflict density feature, a dialogue proportion feature, a relationship density feature and a relationship complexity feature are extracted, wherein the character number feature is the number of character in the preprocessed text to be analyzed, the character personality diversity entropy feature is used to represent the diversity of character personality, the conflict density feature is used to represent the degree of conflict of words in the text to be analyzed, the dialogue proportion feature is the proportion of dialogue in the text to be analyzed, the relationship density feature is used to represent the relationship density between two character entities in the text to be analyzed, and the relationship complexity feature is used to represent the complexity of the relationship connected with the character entity; Step S105, the character number feature, the character personality diversity entropy feature, the conflict density feature, the dialogue proportion feature, the relationship density feature and the relationship complexity feature are subjected to feature processing and fusion to obtain a fused feature vector; Step S106, the fused feature vector is input into an encoder to obtain a preliminary score vector; Step S107, the readability score of the text to be analyzed is determined according to the preliminary score vector.

[0025] In the embodiment, the data preprocessing is performed on the text to be analyzed to obtain a preprocessed text to be analyzed; a plurality of character entity in the preprocessed text to be analyzed is extracted; the preprocessed text to be analyzed is subjected to word segmentation processing to obtain a word sequence, and a dependency syntax tree is constructed according to the word sequence; based on the plurality of character entity, the preprocessed text to be analyzed and the dependency syntax tree, a character quantity feature, a character personality diversity entropy feature, a conflict density feature, a dialogue proportion feature, a relationship density feature and a relationship complexity feature are extracted; the character quantity feature, the character personality diversity entropy feature, the conflict density feature, the dialogue proportion feature, the relationship density feature and the relationship complexity feature are subjected to feature processing and fusion to obtain a fused feature vector; the fused feature vector is input into an encoder to obtain a preliminary score vector; and the readability score of the text to be analyzed is determined according to the preliminary score vector. In this way, the dependency syntax tree is constructed according to the word sequence, which can better analyze the syntax structure of the sentence and lay a good data foundation for later feature extraction; by comprehensively considering the character quantity feature, the character personality diversity entropy feature, the conflict density feature, the dialogue proportion feature, the relationship density feature and the relationship complexity feature, the accuracy of the text readability analysis can be improved, and some features are effectively associated with the actual reading behavior of the reader, thereby improving the explainability of the text readability analysis.

[0026] The data preprocessing of the text to be analyzed can be based on regular expression processing to process the text to be analyzed into a pure text file.

[0027] The plurality of character entity in the preprocessed text to be analyzed can be extracted by entity recognition and co-reference resolution, which can accurately locate the character entity in the text to be analyzed.

[0028] The preprocessed text to be analyzed can be segmented into a sentence sequence by using an existing LTP 4.0 sentence segmentation model, and then segmented into a word sequence by using an existing BERT-wwm model.

[0029] In some embodiments, based on the plurality of character entity, the preprocessed text to be analyzed and the dependency syntax tree, the character quantity feature, the character personality diversity entropy feature, the conflict density feature, the dialogue proportion feature, the relationship density feature and the relationship complexity feature are extracted, including: ; wherein, the character personality diversity entropy feature is represented by the proportion of the character personality label of the i-th character, ​​denotes a summation operation.

[0030] In the embodiment, by analyzing the character personality diversity entropy, the complexity of the character personality in the text to be analyzed can be known, a good data foundation for calculating the readability score of the text in the later stage is laid, and the accuracy of the readability score of the text is improved.

[0031] In some embodiments, based on the multiple character entity, the preprocessed text to be analyzed and the dependency syntax tree, the number of characters, the character personality diversity entropy, the conflict density, the dialogue proportion, the relationship density and the relationship complexity are extracted, including: According to the multiple character entity and the preprocessed text to be analyzed, a character relationship extraction model is used to construct a character relationship graph; The path length relationship feature between two character entities is extracted from the dependency syntax tree; The path length relationship feature and the preprocessed text to be analyzed are fused to obtain an enhanced feature vector; The enhanced feature vector is input into the character relationship extraction model, the relationship triplets are extracted, and the number of extracted relationship triplets is counted to obtain the total number of relationships; The total number of words in the preprocessed text to be analyzed is counted, and the relationship density feature is determined based on the total number of words and the total number of relationships; According to the character relationship graph, the number of relationships connected to each character node is determined; The number of character roles is counted according to the multiple character entities, and the relationship complexity feature is determined based on the number of character roles and the number of relationships connected to each character node.

[0032] In the embodiment, by fusing the path length relationship feature and the preprocessed text to be analyzed, and then extracting the relationship triplets according to the obtained enhanced feature vector, the accuracy of the relationship triplet extraction can be improved, so that the accurate relationship density feature is calculated, and the accurate relationship complexity feature is determined based on the number of character roles and the number of relationships connected to each character node. A good data foundation for calculating the readability score of the text in the later stage is laid, and the accuracy of the readability score of the text is improved.

[0033] The above-mentioned character relationship extraction model can be a character relationship extraction model constructed by using a TPLinker model.

[0034] The path length relationship feature between the two character entities can be extracted from the dependency syntax tree, which can be the number of center words and dependency words or the dependency relationship between the dependency words between the two character entities, and then the path length relationship feature is determined according to the number of dependency relationships. For example, if there are six dependency relationships (i.e., explicit relationship triples) between the center words and the dependency words between the two character entities, and there are five indirect dependency relationships (i.e., non-explicit association triples) between the dependency words and the dependency words, then the number of corresponding dependency relationships between the two character entities is 11, and the number of corresponding dependency relationships between the two character entities is taken as the path length relationship feature.

[0035] The path length relationship feature and the preprocessed text to be analyzed can be fused to obtain an enhanced feature vector, which can be fused by splicing, adding or attention mechanism, etc. The path length relationship feature and the preprocessed text to be analyzed are fused to obtain an enhanced feature vector.

[0036] The enhanced feature vector can be input into the role relationship extraction model to extract relationship triples, which can be input into the role relationship extraction model to extract explicit relationship triples and non-explicit association triples.

[0037] In some embodiments, based on the multiple character entities, the preprocessed text to be analyzed and the dependency syntax tree, the role quantity feature, the role character diversity entropy feature, the conflict density feature, the dialogue proportion feature, the relationship density feature and the relationship complexity feature are extracted, including: The speaker is parsed and associated with the speaker. The dialogue content of the speaker in the preprocessed text to be analyzed is located by dialogue boundary identification; The number of dialogue words in the dialogue content is counted, and the total number of words in the preprocessed text to be analyzed is counted; The dialogue proportion feature is determined according to the number of dialogue words and the total number of words.

[0038] In this embodiment, by determining the dialogue proportion feature, the actual reading behavior of the reader can be effectively associated, thereby laying a good data foundation for calculating the readability score of the text in the later stage, so as to improve the accuracy of the readability score of the text.

[0039] The speaker analysis and dialogue boundary identification are both technologies in natural language processing, and this embodiment does not make specific description.

[0040] In some embodiments, based on multiple character entities, preprocessed text to be analyzed, and dependency syntax trees, character quantity features, character personality diversity entropy features, conflict density features, dialogue ratio features, relationship density features, and relationship complexity features are extracted, including: ; in, represents the conflict density feature, Indicates the number of all conflicts in the text to be analyzed after preprocessing. Indicates the The conflict type weight of the conflict words, Indicates the The emotional emphasis of the conflicting words, Indicates the total number of words in the preprocessed text to be analyzed.

[0041] In this embodiment, by calculating the conflict density feature, it is possible to better effectively associate it with the reader's actual reading behavior, thereby laying a good data foundation for the subsequent calculation of the readability score of the text, thereby improving the accuracy of the readability score of the text.

[0042] In some embodiments, the fused feature vector is input into an encoder to obtain a preliminary score vector, including: The self-attention mechanism in the encoder captures the interaction between features in the fused feature vector and assigns attention weights to features with interaction relationships; According to the assigned attention weights, the fused feature vectors are integrated through the feedforward network and layer normalization in the encoder to obtain a preliminary score vector.

[0043] In this embodiment, by assigning attention weights to features with interactive relationships, the model can focus on key related features, thereby improving the accuracy of the text readability score.

[0044] The above encoder may be a Transformer encoder.

[0045] In some implementations, determining a readability score of the text to be analyzed based on the preliminary score vector includes: Input the preliminary rating vector into the fully connected layer to obtain the output of the fully connected layer; The output of the fully connected layer is mapped through the activation function to obtain the readability score of the text to be analyzed.

[0046] In this embodiment, the readability score of the text to be analyzed is obtained by using an accurate preliminary score vector, which can improve the accuracy of the readability score of the text.

[0047] For the convenience of those skilled in the art to understand, a set of best embodiments is provided below: Text readability analysis is a core topic in the intersection of natural language processing (NLP) and educational technology, aiming to evaluate the reading difficulty and understanding efficiency of a text through quantitative indicators. Traditional readability assessment tools (such as Flesch-Kincaid and SMOG index) mainly rely on surface language features (such as sentence length, number of phonemes, and vocabulary complexity, etc.), although widely used in basic education and document simplification, they have significant limitations, mainly in the absence of semantic depth, ignoring narrative structure, character interaction, and emotional tension, and cannot distinguish between "simple syntax but complex plot" and "complex syntax but straightforward logic" texts.

[0048] The prior art has the problems of low accuracy and poor interpretability of text readability analysis, because most of the existing technical methods focus on a single dimension (such as only analyzing sentiment or syntax), lacking systematic integration of multi-modal features; the readability score is not effectively associated with the actual reading behavior of the reader, resulting in insufficient model interpretability.

[0049] To solve the problems existing in the prior art, the technical scheme of the embodiment includes the following contents: With reference to Figure 2 The embodiment adopts a modular pipeline architecture, divided into data preprocessing and segmentation layer, parallel feature extraction layer, and comprehensive evaluation layer, supporting high concurrency processing and module independent expansion performance. Specifically, it includes: (1) Data preprocessing and segmentation layer.

[0050] With reference to Figure 3 This step is the core basic module of the natural language processing (NLP) pipeline, aiming to convert raw unstructured text into standardized, semantically clear structured data, providing high-quality input for subsequent feature extraction.

[0051] The text cleaning stage is to delete HTML tags and special symbols through regular expression matching, and replace full-width punctuation with half-width punctuation based on rules. The output is a UTF-8 encoded pure text file, with noise character proportion less than 0.1%, obtaining the preprocessed text (i.e. the text to be analyzed).

[0052] Sentence segmentation and tokenization (Sentence Segmentation and Tokenization) are performed on the preprocessed text: continuous text is divided into sentences and word units. Specifically, based on the sentence segmentation model of LTP 4.0 (F1=0.94), complex sentence patterns containing ellipsis are supported, and then the BERT-wwm model is used for adaptive segmentation of the segmented sentences to obtain the word sequence.

[0053] Named Entity Recognition and Coreference Resolution (NER and Coreference Resolution): used to identify named entities in text and resolve reference ambiguity. Specifically, a sequence labeling model based on SpanBERT can obtain entity types such as person, place, organization, and time, but this embodiment only uses person character entities. Coreference resolution uses SpanBERT combined with the OntoNotes (OntoNotes is a large-scale multilingual corpus) dataset for training to improve the accuracy of person character alias resolution and solve long-distance reference problems. Through the named entity recognition (NER) model SpanBERT, the person character name can be accurately located, and the recognition recall rate can be improved by combining a domain dictionary.

[0054] Dependency Syntax Analysis: used to parse the grammatical structure of a sentence, extract core components such as subject, verb, and object, and represent them as a dependency syntax tree. The main process is: 1. Tokenization and part-of-speech tagging, which divides the sentence into a sequence of words (uses the sentence segmentation and tokenization method in the previous section to obtain the word sequence). Each word in the word sequence is tagged with its part-of-speech (noun, verb, adjective, etc.); 2. Dependency relationship identification determines whether there is a dependency relationship between all word pairs in the sentence and what type of dependency relationship exists, and finally outputs a set of dependency relationship triples (including dependent word, relationship type, and center word). Common dependency syntax analysis methods include rule-based methods, statistical methods, or neural network-based methods. Using a neural network-based method, the self-attention mechanism of Transformer is used to automatically learn distributed representations (word vectors) of words and part-of-speech tags, and these rich distributed representations are used to predict dependency relationships.

[0055] (2) Parallel feature extraction layer.

[0056] The core function of the role information extraction module is to analyze the personality of the characters in the text (i.e., the preprocessed text to be analyzed), and 45 types of character personalities are preset. The large language model (LLM) is used to classify and count the proportion of character personalities .

[0057] All character personality diversity entropy, formula: ; Where, represents the proportion of the th character personality label , represents the sum, i.e., the total character personality entropy, and the character personality diversity entropy is obtained.

[0058] The character relationship extraction module (i.e., the TPLinker model) aims to build an interaction network (i.e., a character relationship graph) between character entities, and quantify the dependency relationship type and strength. The module implements end-to-end relationship joint extraction based on the TPLinker model, which can simultaneously identify entity pairs and their relationship types, solving the error accumulation problem of the traditional Pipeline method.

[0059] The relationship density is defined as the number of relationships per unit of text length: ; Wherein, is the total number of relationships (the number of relationship triples obtained by the TPLinker model and the number of triples associated non-explicitly, the total number of relationship triples ), is the total number of words in the text.

[0060] Specifically, the dependency syntax tree of the word sequence is constructed using the previous Transformer-based dependency syntax analyzer. Each word has a center word, a dependency relationship type, and a dependent word. The tree structure of this dependency syntax tree represents the syntactic relationship between words in the sentence.

[0061] The dependency syntax tree is converted into features that can be used by the relationship joint extraction model (i.e., the TPLinker model). The path length relationship feature between two character entities (e1 and e2) on the dependency syntax tree is directly extracted from the dependency syntax tree as a discrete feature, i.e., the two character entities contain multiple dependent words and center words, and there is a path between the two connecting words. The number of paths between these words is counted, which is the path length relationship feature. The path length relationship feature is input into the relationship joint extraction model for relationship triple extraction.

[0062] The original sentence text (i.e., the preprocessed text) and the generated dependency information representation (i.e., the discrete feature) are concatenated, added, or fused through an attention mechanism to obtain an enhanced input representation (i.e., an enhanced feature vector). After inputting the enhanced input representation into the relationship joint extraction (TPLinker) model, the TPLinker model marks the start and end positions of the character entities, marks the start and end positions of the two character entities in the relationship triple, and predicts the relationship type between the character entity pairs (the relationship type refers to explicit relationship triples and triples associated non-explicitly), thereby obtaining all extracted relationship triples. The number of extracted relationship triples is counted to obtain the total number of relationships M.

[0063] The network average degree is defined as the average number of connections of the nodes in the character relationship graph to reflect the complexity of the relationship: ; wherein, the number of relationships connected to each character node (i.e. each character entity) (counting all triples from the character node directly from the character relation graph), the number of characters in the text (the number of characters involved in the text as extracted by the LLM).

[0064] The dialogue detection module is responsible for locating dialogue content in the text and associating the speakers, and its technical implementation is divided into two stages of dialogue boundary recognition and speaker analysis (both of which are technologies in natural language processing, and the present embodiment does not make specific descriptions). The module outputs structured dialogue records (i.e. dialogue content), including speakers, content text and sentiment polarity (used for cross-validation with the emotion module), while also counting dialogue proportion indicators (i.e. the number of dialogue words / total number of words in the text) and identifying dialogue-intensive paragraphs.

[0065] The event conflict extraction module focuses on identifying antagonistic events in the text and evaluating their narrative impact. This module uses a multi-level processing flow: first, a predefined conflict trigger word library (including words such as "struggle" and "betrayal") and a BERT-CRF model are used to label conflict event boundaries (i.e. conflict words), and the conflict trigger word library contains conflict type weights (which are also self-defined), such as "betrayal" and "war" with higher conflict type weights. According to the labeled conflict event boundaries, the text is input into the Transformer model, and the sentiment intensity of the conflict words is output, then normalized by Softmax, and then the semantic role labeling is used to determine the conflict subject and type (the purpose is to obtain the conflict type weight). The output results include conflict density (times per thousand characters), type distribution and intensity curve, providing key tension indicators for readability analysis.

[0066] The event conflict quantification model defines the conflict density formula as: ; wherein, is the conflict type weight, is the sentiment intensity, is the total number of words in the text, is the total number of conflicts in the text (i.e. the number of conflict words).

[0067] (3) Scoring model layer (i.e. comprehensive evaluation layer).

[0068] The scoring model layer serves as the decision-making center of the system, responsible for concatenating and fusing the multi-dimensional features extracted by each module into a unified readability score, and its technical implementation covers feature fusion and dynamic weight distribution.

[0069] Feature input: The input feature vector is composed of six indicators obtained by four modules, including the number of characters, character personality diversity entropy, conflict density, dialogue proportion, relationship density, and relationship complexity. The character diversity entropy and conflict density indicators are normalized by Z-score to eliminate dimensional differences. For long-tail distribution indicators (such as the number of characters), Box-Cox transformation is used to improve normality.

[0070] Feature embedding: The normalized or Box-Cox transformed feature vector (including the number of characters, character personality diversity entropy, conflict density, dialogue proportion, relationship density, and relationship complexity) is spliced, and a structured feature vector is generated to obtain a fused feature vector.

[0071] The preprocessed six indicators are converted into structured feature embedding vectors (i.e., fused feature vectors) that can be processed by the model. Specifically, the numerical value of each indicator is mapped to a fixed-dimensional vector through an embedding layer, and finally a feature sequence with a length of 6 is formed (each position corresponds to an embedding vector of an indicator).

[0072] Transformer encoder: feature interaction modeling and preliminary scoring. The Transformer encoder, as the core module, performs deep modeling on the feature sequence. Specifically: Self-attention mechanism: through multi-head self-attention calculation, the complex interaction relationships between the six indicators are captured (for example: the correlation between the number of characters and the relationship density, the synergistic effect between conflict density and dialogue proportion, the mutual influence between emotional entropy and character personality diversity entropy, etc.).

[0073] Dynamic weight allocation: based on the correlation between features (such as "high conflict density + high dialogue proportion" having a more significant impact on readability), automatically allocate attention weights for different feature interactions, and focus on key associated features.

[0074] Output preliminary score: according to the allocated attention weights, integrate the modeled feature sequence through the encoder's feedforward network and layer normalization, and output a preliminary score vector with a dimension of 1 (reflecting the basic impact of feature interaction on readability).

[0075] Score integration: final readability score generation.

[0076] The preliminary score vector output by the Transformer encoder is input into the fully connected layer (i.e., linear transformation layer), and the vector is mapped to the final text readability score (i.e., the readability score of the text to be analyzed) through the activation function (ReLU) (the numerical range is set to 0-10).

[0077] Compared with the prior art, the technical scheme of the embodiment has the following advantages: 1、The scoring model layer realizes high-precision and high-efficiency readability evaluation by dynamically fusing structured features and semantic features. Experiments show that the system model in this embodiment performs stably in cross-genre scenarios, with an MAE as low as 2.9 points, and has strong interpretability (SHAP value analysis).

[0078] 2、By integrating four core modules of role information extraction, role relationship extraction, dialogue detection and event conflict extraction, a multi-dimensional text evaluation system is constructed to finally generate readability scores. This framework not only focuses on traditional readability indicators (such as sentence length and lexical complexity), but also introduces deep semantic features such as narrative structure, emotional tension and character interaction, suitable for various text types such as novels, scripts and news. The research goal is to solve the problem that existing readability evaluation tools ignore the internal logic and emotional expression of the text, and to provide more comprehensive analysis support for education, publishing and human-computer interaction fields.

[0079] Reference Figure 4 The embodiment of the application also provides a text readability analysis system based on NLP processing, which comprises a data processing unit 401, an entity extraction unit 402, a syntax tree construction unit 403, a feature extraction unit 404, a feature fusion unit 405, a scoring vector obtaining unit 406 and a readability scoring unit 407, wherein: The data processing unit 401 is used for data preprocessing of the text to be analyzed to obtain the preprocessed text to be analyzed; The entity extraction unit 402 is used for extracting multiple character role entities in the preprocessed text to be analyzed; The syntax tree construction unit 403 is used for performing word segmentation processing on the preprocessed text to be analyzed to obtain a word sequence, and constructing a dependency syntax tree according to the word sequence; The feature extraction unit 404 is used for extracting a role quantity feature, a role personality diversity entropy feature, a conflict density feature, a dialogue proportion feature, a relationship density feature and a relationship complexity feature based on the multiple character role entities, the preprocessed text to be analyzed and the dependency syntax tree, wherein the role quantity feature is the number of character roles in the preprocessed text to be analyzed, the role personality diversity entropy feature is used for representing the diversity of the personality of the character, the conflict density feature is used for representing the degree of conflict of the words in the text to be analyzed, the dialogue proportion feature is the proportion of dialogue in the text to be analyzed, the relationship density feature is used for representing the relationship density between two character role entities in the text to be analyzed, and the relationship complexity feature is used for representing the complexity of the relationship connected with the character role entity; The feature fusion unit 405 is used for feature processing and fusion of the role quantity feature, the role personality diversity entropy feature, the conflict density feature, the dialogue proportion feature, the relationship density feature and the relationship complexity feature to obtain a fused feature vector. The scoring vector obtaining unit 406 is configured to input the fused feature vector into an encoder to obtain a preliminary scoring vector. The readability scoring unit 407 is configured to determine a readability score of the text to be analyzed according to the preliminary scoring vector.

[0080] It should be noted that, since the text readability analysis system based on NLP processing in the embodiment and the text readability analysis method based on NLP processing described above are based on the same inventive concept, the corresponding content in the method embodiment is also applicable to the system embodiment, and will not be described in detail here.

[0081] With reference to Figure 5 The embodiment of the present application further provides an electronic device, and the electronic device comprises: at least one memory; at least one processor; at least one program; The program is stored in the memory, and the processor executes the at least one program to implement the text readability analysis method based on NLP processing provided in the embodiment of the present disclosure.

[0082] The electronic device can be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a vehicle-mounted computer, etc.

[0083] The electronic device of the embodiment of the present application will be described in detail below.

[0084] The processor 1600 can be implemented in the form of a general central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits, etc., and is configured to execute a related program to implement the technical solutions provided in the embodiment of the present disclosure. The memory 1700 can be implemented in the form of a read only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM), etc. The memory 1700 can store an operating system and other application programs, and when the technical solutions provided in the embodiment of the present disclosure are implemented by software or firmware, the related program codes are saved in the memory 1700 and are called and executed by the processor 1600 to implement the text readability analysis method based on NLP processing in the embodiment of the present disclosure.

[0085] The input / output interface 1800 is configured to realize information input and output. The communication interface 1900 is configured to realize communication interaction between the device and other devices. The communication can be realized in a wired manner (for example, a USB, a network cable, or the like) or in a wireless manner (for example, a mobile network, WIFI, Bluetooth, or the like). The bus 2000 is configured to transmit information between various components (for example, the processor 1600, the memory 1700, the input / output interface 1800, and the communication interface 1900) of the device. The processor 1600, the memory 1700, the input / output interface 1800, and the communication interface 1900 are connected to each other through the bus 2000 to realize communication connection between the device.

[0086] The disclosure also provides a storage medium, which is a computer readable storage medium, and stores computer executable instructions for causing a computer to execute the above-mentioned NLP processing based text readability analysis method.

[0087] The memory is a non-transitory computer readable storage medium, and can be used to store a non-transitory software program and a non-transitory computer executable program. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, for example, at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and the remote memory can be connected to the processor through a network. Examples of the network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0088] The embodiments described in the disclosure are used to more clearly illustrate the technical solutions of the disclosure, and do not constitute a limitation on the technical solutions provided by the disclosure. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the disclosure are also applicable to similar technical problems.

[0089] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the disclosure, and can include more or fewer steps than the figures, or combine certain steps or different steps.

[0090] The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separate, that is, can be located in one place or distributed on multiple network units. According to actual needs, part or all of the modules can be selected to realize the purposes of the embodiments.

[0091] Those skilled in the art can understand that all or some of the steps in the method disclosed above, the functional modules / units in the system and the device can be implemented by software, firmware, hardware or a proper combination thereof.

[0092] The terms "first", "second", "third", "fourth" and the like in the description of the application and in the claims hereof, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so termed herein is only to distinguish one feature from another and is not meant to or does imply preference or inherent superiority of the elements over or relative to each other or of the incipient order. Embodiments of the application described herein can be implemented in a chronological order, in reverse order or in any suitable order.

[0093] It should be understood that, in the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the relationship between associated objects, which means that there can be three relationships, for example, "A and / or B" can mean that there are three cases of only A, only B and A and B at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or the like means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0094] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed objects can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0095] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e., may be located in one place, or may be distributed to multiple network units. Part or all of the units may be selected according to actual needs to achieve the purpose of the embodiment.

[0096] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can be physically present alone, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0097] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes multiple instructions for causing an electronic device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method of each embodiment of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media. The above combines the drawings to make a detailed description of the embodiments of the present application, but the present application is not limited to the above embodiments, and various changes can be made within the knowledge range of ordinary skilled persons in the art without departing from the purpose of the present application.

[0098] The above combines the drawings to make a detailed description of the embodiments of the present application, but the present application is not limited to the above embodiments, and various changes can be made within the knowledge range of ordinary skilled persons in the art without departing from the purpose of the present application.

Claims

1. A text readability analysis method based on NLP processing, characterized in that: The method comprises: Perform data preprocessing on the text to be analyzed to obtain the preprocessed text to be analyzed; Extracting multiple character entities from the preprocessed text to be analyzed; Performing word segmentation on the preprocessed text to be analyzed to obtain a word sequence, and constructing a dependency syntax tree based on the word sequence; Based on the multiple character role entities, the preprocessed text to be analyzed and the dependency syntax tree, extracting a character quantity feature, a character personality diversity entropy feature, a conflict density feature, a dialogue ratio feature, a relationship density feature and a relationship complexity feature, wherein the character quantity feature is the statistical number of characters in the preprocessed text to be analyzed, the character personality diversity entropy feature is used to characterize the diversity of character personalities, the conflict density feature is used to characterize the degree of conflict between words in the text to be analyzed, the dialogue ratio feature is the proportion of dialogues in the text to be analyzed, the relationship density feature is used to characterize the relationship density between two character role entities in the text to be analyzed, and the relationship complexity feature is used to characterize the complexity of the relationship connected to the character role entity; Processing and fusing the character quantity feature, the character personality diversity entropy feature, the conflict density feature, the conversation ratio feature, the relationship density feature, and the relationship complexity feature to obtain a fused feature vector; Inputting the fused feature vector into an encoder to obtain a preliminary score vector; The readability score of the text to be analyzed is determined according to the preliminary score vector.

2. The text readability analysis method based on NLP processing according to claim 1 is characterized in that The extracting of character quantity features, character personality diversity entropy features, conflict density features, dialogue proportion features, relationship density features, and relationship complexity features based on the multiple character role entities, the preprocessed text to be analyzed, and the dependency syntax tree includes: ； in, Represents the character diversity entropy characteristics, Indicates the Character personality tags The proportion of Represents a sum operation.

3. The text readability analysis method based on NLP processing according to claim 1, characterized in that: The extracting of character quantity features, character personality diversity entropy features, conflict density features, dialogue proportion features, relationship density features, and relationship complexity features based on the multiple character role entities, the preprocessed text to be analyzed, and the dependency syntax tree includes: Constructing a role relationship graph using a role relationship extraction model based on the multiple character role entities and the preprocessed text to be analyzed; Extracting a path length relationship feature between two persona entities from the dependency syntax tree; Fusing the path length relationship feature with the preprocessed text to be analyzed to obtain an enhanced feature vector; Inputting the enhanced feature vector into the role relationship extraction model to extract relationship triples, and counting the number of extracted relationship triplets to obtain the total number of relationships; Counting the total number of words in the preprocessed text to be analyzed, and determining a relationship density feature based on the total number of words and the total number of relationships; Determining the number of relationships connected to each character node based on the character relationship graph; The number of personas is counted according to the plurality of persona entities, and a relationship complexity feature is determined based on the number of personas and the number of relationships connected to each persona node.

4. The text readability analysis method based on NLP processing according to claim 1, characterized in that: The extracting of character quantity features, character personality diversity entropy features, conflict density features, dialogue proportion features, relationship density features, and relationship complexity features based on the multiple character role entities, the preprocessed text to be analyzed, and the dependency syntax tree includes: Associating speakers through speaker resolution; Locating the conversation content of the speaker in the preprocessed text to be analyzed by dialogue boundary recognition; Counting the number of words in the conversation content, and counting the total number of words in the pre-processed text to be analyzed; A conversation proportion feature is determined according to the conversation word count and the total word count.

5. The text readability analysis method based on NLP processing according to claim 1 is characterized in that: The extracting of character quantity features, character personality diversity entropy features, conflict density features, dialogue proportion features, relationship density features, and relationship complexity features based on the multiple character role entities, the preprocessed text to be analyzed, and the dependency syntax tree includes: ； in, represents the conflict density feature, represents the number of all conflicts in the text to be analyzed after the preprocessing, Indicates the The conflict type weight of the conflict words, Indicates the The emotional emphasis of the conflicting words, Indicates the total number of words in the preprocessed text to be analyzed.

6. The text readability analysis method based on NLP processing according to claim 1, characterized in that: The step of inputting the fused feature vector into an encoder to obtain a preliminary score vector comprises: Capturing the interactive relationship between features in the fused feature vector through the self-attention mechanism in the encoder, and assigning attention weights to features with interactive relationships; According to the assigned attention weights, the fused feature vectors are integrated through the feedforward network and layer normalization in the encoder to obtain a preliminary score vector.

7. The text readability analysis method based on NLP processing according to claim 1 is characterized in that: Determining the readability score of the text to be analyzed based on the preliminary score vector includes: Inputting the preliminary score vector into the fully connected layer to obtain the output result of the fully connected layer; The output of the fully connected layer is mapped through the activation function to obtain the readability score of the text to be analyzed.

8. A text readability analysis system based on NLP processing, characterized in that: The system comprises: A data processing unit, configured to perform data preprocessing on the text to be analyzed to obtain the preprocessed text to be analyzed; An entity extraction unit, configured to extract a plurality of character role entities from the preprocessed text to be analyzed; A syntax tree construction unit is used to perform word segmentation on the preprocessed text to be analyzed to obtain a word sequence, and to construct a dependency syntax tree based on the word sequence; a feature extraction unit for extracting, based on the multiple character entities, the preprocessed text to be analyzed, and the dependency syntax tree, a character quantity feature, a character personality diversity entropy feature, a conflict density feature, a dialogue ratio feature, a relationship density feature, and a relationship complexity feature, wherein the character quantity feature is the statistical number of characters in the preprocessed text to be analyzed, the character personality diversity entropy feature is used to characterize the diversity of character personalities, the conflict density feature is used to characterize the degree of conflict between words in the text to be analyzed, the dialogue ratio feature is the proportion of dialogues in the text to be analyzed, the relationship density feature is used to characterize the relationship density between two character entities in the text to be analyzed, and the relationship complexity feature is used to characterize the complexity of the relationship connected to the character entity; a feature fusion unit, configured to perform feature processing and fusion on the character quantity feature, the character personality diversity entropy feature, the conflict density feature, the conversation proportion feature, the relationship density feature, and the relationship complexity feature to obtain a fused feature vector; a scoring vector obtaining unit, configured to input the fused feature vector into an encoder to obtain a preliminary scoring vector; The readability scoring unit is used to determine the readability score of the text to be analyzed based on the preliminary scoring vector.

9. An electronic device, characterized in that: It includes at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor to enable the at least one control processor to perform the text readability analysis method based on NLP processing as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the text readability analysis method based on NLP processing according to any one of claims 1 to 7.

Images