Semantic understanding and automatic abstract generation system for literature works

Abstract

The invention relates to the field of artificial intelligence, in particular to a semantic understanding and automatic abstract generation system for literature works, which comprises a text collection module, a semantic understanding module, a theme extraction module, an abstract generation module and an optimization module. Performing semantic analysis on the literature data set in combination with hierarchical aggregation, multi-level information fusion and a knowledge graph, performing deep pre-training on text data through a RoBERTa model, and capturing a complex semantic relationship in the literature; the method comprises the following steps: pre-training a semantic understanding model by adopting a shielded language model, optimizing the shielded language model by adopting a DTO-PSO hybrid optimization algorithm, and introducing an information entropy feedback mechanism and a constraint processing mechanism to improve the depth and accuracy of semantic understanding.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a system for semantic understanding and automatic summary generation of literary works. Background Art

[0002] With the advancement of artificial intelligence, semantic understanding and automatic summary generation technologies are gradually applied to various text analysis fields, especially in the semantic analysis of literary works. However, general semantic understanding methods have problems such as incomplete semantic understanding, poor summary quality, information loss, lack of deep emotional and contextual analysis, and insufficient understanding of complex semantic relationships and contextual dependencies in literary works, resulting in superficial or one-sided semantic analysis, and failure to effectively extract and aggregate information in a hierarchical manner, resulting in loss or duplication of important information; ordinary semantic understanding models have problems such as low semantic understanding accuracy, poor summary diversity, and poor logic, and are unable to well understand the complex semantics of metaphors, symbols, and implicit emotions in literary works, resulting in the generated summary deviating from the core theme of the literary works. Insufficient exploration during the optimization process leads to a lack of diversity in the generated summary, and a longer time is required for parameter adjustment and optimization during the training process, resulting in inefficient summary generation and slower system response. Summary of the invention

[0003] In view of the above situation, the present invention provides a semantic understanding and automatic summary generation system for literary works. In view of the problems of incomplete semantic understanding, poor summary quality, information loss, and lack of deep emotion and context analysis in general semantic understanding methods, the present invention adopts the RoBERTa model in the semantic understanding module, combines hierarchical aggregation, multi-level information fusion and knowledge graph to perform semantic analysis on the literary work data set, and uses the RoBERTa model to perform deep pre-training on text data to capture the complex semantic relationships in literary works. Combined with hierarchical aggregation, the content of long literary works is refined in a hierarchical manner, thereby avoiding information loss or one-sided analysis, and using knowledge graphs to improve the understanding of roles, events and time in literary works. The ability to understand the relationship between entities; in view of the problems of low semantic understanding accuracy, poor summary diversity and poor logic in ordinary semantic understanding models, the present invention adopts a masked language model to pre-train the semantic understanding model, and at the same time adopts a DTO-PSO hybrid optimization algorithm to optimize the masked language model, introduces an information entropy feedback mechanism and a constraint processing mechanism to improve the depth and accuracy of semantic understanding, evaluates the state of the particle swarm through information entropy, finds a balance between exploring new semantic structures and refining existing semantic understanding, refines the optimization results through local search in the global search process, extracts important information from a wider range of texts through the optimized masked language model and information entropy regulation, and generates accurate summaries in combination with semantic understanding.

[0004] The present invention provides a system for semantic understanding and automatic summary generation of literary works, including a text collection module, a semantic understanding module, a topic extraction module, a summary generation module and an optimization module, which specifically includes the following contents:

[0005] The text collection module collects text data, including various literary works, and performs data cleaning, word segmentation and part-of-speech tagging;

[0006] The semantic understanding module constructs a data set of literary works, uses the RoBERTa model, combines hierarchical aggregation, multi-level information fusion and knowledge graph to perform semantic analysis on the data set of literary works, and obtains the deep meaning of literary works;

[0007] The topic extraction module uses a clustering algorithm to extract topics and key concepts from literary works;

[0008] The summary generation module automatically generates a summary of a literary work by combining the deep meaning, theme and key concepts of the literary work based on a sequence-to-sequence model;

[0009] The optimization module collects user feedback, performs manual corrections, and continuously optimizes.

[0010] Furthermore, semantic analysis is performed in the semantic understanding module, which specifically includes the following steps:

[0011] Step S1: text data preparation, preprocessing the text data, including noise removal and text segmentation, obtaining the processed text data and organizing it into a literature data set;

[0012] Step S2: Model construction: construct a model based on RoBERTa, add a hierarchical aggregation layer and a knowledge graph enhancement layer to obtain a semantic understanding model;

[0013] Step S3: Model pre-training: using a masked language model to pre-train the semantic understanding model, and setting up paragraph aggregation prediction simultaneously;

[0014] Step S4: Model adjustment: using the literature data set to train and adjust the semantic understanding model to obtain an adjusted semantic understanding model;

[0015] Step S5: semantic analysis, using the adjusted semantic understanding model to perform semantic analysis and generate a deep semantic representation of the text;

[0016] Furthermore, step S2 specifically includes the following contents:

[0017] Input embedding layer: The RoBERTa model is selected as the basic model, and the literature dataset is tokenized. The text data in the literature dataset is split into subword fragments using subword segmentation. The subword fragments are converted into vocabulary index format and passed into the RoBERTa model. The deep semantic features of the literature dataset are extracted through a multi-layer Transformer architecture to obtain a chapter-level representation.

[0018] Encoding layer: Based on the Transformer architecture, it includes 24 self-attention mechanism layers and a feedforward neural network

[0019] Hierarchical aggregation layer: The text is processed in a hierarchical manner, and the text data is divided into three levels: sentence, paragraph, and chapter. The sentences are encoded using Bi-LSTM, and the semantic representation of each sentence is extracted. The sentence representations are aggregated at the paragraph level, and the weight distribution between sentences is calculated through the self-attention mechanism to generate the global representation of the paragraph. At the chapter level, the paragraph representation is further aggregated and optimized through the hierarchical attention mechanism to generate the semantic representation of the entire text.

[0020] Knowledge graph fusion layer: Based on the external knowledge graph, the RotaE model is used to build entity embeddings. The fusion strategy of the attention mechanism is used to weightedly fuse entity embeddings with deep semantic features to generate semantic representations with background information.

[0021] Output layer: receives the semantic representation of the hierarchical aggregation layer and the knowledge graph fusion layer, and maps it into vector output;

[0022] Further, step S3 specifically includes the following steps:

[0023] Step S31: masking language model setting, select 15% of the words in the literature data set to mask and replace them with MASK. The goal of the masking language model is to predict the masked words;

[0024] Step S32: Setting paragraph aggregation tags: setting a target tag for each input paragraph through an external knowledge graph. These target tags are called paragraph aggregation tags.

[0025] Step S33: model input, inputting the masked literature dataset and paragraph aggregation labels into the masked language model;

[0026] Step S34: model training, using the cross entropy loss function to predict the masked words and paragraph aggregation labels, and combining the two prediction results into a comprehensive loss function;

[0027] Step S35: Model optimization, using the DTO-PSO hybrid optimization algorithm to optimize the masked language model and update the model parameters;

[0028] Furthermore, in step S35, the DTO-PSO hybrid optimization algorithm is used to train the masked language model, which specifically includes the following steps:

[0029] Step S351: Initialize the particle swarm, randomly generate the position of particles, each particle represents a parameter solution of the masking language model, introduce an adaptive weighting factor, define a feasible solution space, check whether the position of each particle is in the feasible solution space, and if it is not in the feasible solution space, use the repair mechanism to return to the feasible solution space. The feasible solution space is as follows: ;

[0030] in, is the feasible solution space, is the real number space, is a variable, is an inequality constraint, is an equality constraint, for The index of for The index of

[0031] Step S352: Introduce an information entropy feedback mechanism to evaluate the particle swarm through information entropy. The information entropy formula is: ;

[0032] in, For the During the round iteration, the information entropy of the particle swarm is is the total number of particles in the particle swarm, For the The particle in Probability distribution of fitness in iterations;

[0033] When the information entropy decreases and the particle swarm tends to concentrate, the exploration range of the search area is increased. When the information entropy is low, the particle speed is increased. The formula for updating the particle speed is: ;

[0034] in, It is The particle in The speed in the iteration, It is The particle in The speed in iterations;

[0035] Introduce the constraint processing mechanism, calculate the particle fitness, and introduce the penalty function into the original objective function. The formula used is as follows: ;

[0036] in, is the fitness function after constraint processing, is the original objective function, is an inequality constraint, is an equality constraint, and is the penalty coefficient;

[0037] Step S353: Use the PSO formula to update the particle position and velocity. The formula used is as follows: ; ;

[0038] in, is the inertia weight, , are individual learning factors and social learning factors, For the The particle in The optimal position of an individual in the iteration, For the The global optimal position in the iteration, For the The particle in The current position in iterations;

[0039] Constraint mechanism processing, after updating the position, check the particle Whether the constraints of the feasible solution space are met. If not, a penalty mechanism is applied to modify the fitness;

[0040] Step S354: Introduce DTO local search. After each particle update, use DTO to perform local optimization on the particle. The formula used is as follows: ;

[0041] in, It is The new position of a particle after local search, It is The particle in The initial position in the iteration, is the step size coefficient, The dimension is A random vector of ;

[0042] Constraint processing, DTO local search, check particles Whether the constraints of the feasible solution space are met. If not, a penalty mechanism is applied to modify the fitness;

[0043] Step S355: Global and local optimal update, the formula used is as follows: ; ;

[0044] in, It is The particle in The optimal position of the new individual after iterations is: is the fitness function, For the The global optimal position after iterations;

[0045] Step S356: Output the result. Output, as the optimal parameter solution for the masked language model.

[0046] The beneficial effects achieved by the present invention using the above scheme are as follows:

[0047] (1) In view of the above situation, the present invention provides a semantic understanding and automatic summary generation system for literary works. In view of the problems of incomplete semantic understanding, poor summary quality, information loss, and lack of deep emotional and contextual analysis in general semantic understanding methods, the present invention adopts the RoBERTa model in the semantic understanding module, combines hierarchical aggregation, multi-level information fusion and knowledge graph to perform semantic analysis on the literary work data set, and uses the RoBERTa model to perform deep pre-training on text data to capture the complex semantic relationships in literary works. Combined with hierarchical aggregation, the content of long literary works is refined in a hierarchical manner, thereby avoiding information loss or one-sided analysis, and using knowledge graph to improve the ability to understand the relationships between characters, events and timeline entities in literary works;

[0048] (2) In view of the problems of low semantic understanding accuracy, poor summary diversity and poor logic in ordinary semantic understanding models, the present invention adopts a masked language model to pre-train the semantic understanding model, and adopts a DTO-PSO hybrid optimization algorithm to optimize the masked language model, introduces an information entropy feedback mechanism and a constraint processing mechanism to improve the depth and accuracy of semantic understanding, evaluates the state of the particle swarm through information entropy, finds a balance between exploring new semantic structures and refining existing semantic understanding, refines the optimization results through local search during the global search process, extracts important information from a wider range of texts through the optimized masked language model and information entropy regulation, and generates accurate summaries in combination with semantic understanding. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A schematic diagram of a system for semantic understanding and automatic summary generation of literary works provided by the present invention.

[0050] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0052] In the description of the present invention, it should be understood that terms such as “upper”, “lower”, “front”, “back”, “left”, “right”, “top”, “bottom”, “inside” and “outside” indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as limiting the present invention.

[0053] Example 1, see Figure 1 The present invention provides a system for semantic understanding and automatic summary generation of literary works, including a text collection module, a semantic understanding module, a topic extraction module, a summary generation module and an optimization module, which specifically includes the following contents:

[0054] The text collection module collects text data, including various literary works, and performs data cleaning, word segmentation and part-of-speech tagging;

[0055] The semantic understanding module constructs a data set of literary works, uses the RoBERTa model, combines hierarchical aggregation, multi-level information fusion and knowledge graph to perform semantic analysis on the data set of literary works, and obtains the deep meaning of literary works;

[0056] The topic extraction module uses a clustering algorithm to extract topics and key concepts from literary works;

[0057] The summary generation module automatically generates a summary of a literary work by combining the deep meaning, theme and key concepts of the literary work based on a sequence-to-sequence model;

[0058] The optimization module collects user feedback, performs manual corrections, and continuously optimizes.

[0059] Embodiment 2: This embodiment is based on the above embodiment and performs semantic analysis in a semantic understanding module, specifically comprising the following steps:

[0060] Step S1: text data preparation, preprocessing the text data, including noise removal and text segmentation, obtaining the processed text data and organizing it into a literature data set;

[0061] Step S2: Model construction: construct a model based on RoBERTa, add a hierarchical aggregation layer and a knowledge graph enhancement layer to obtain a semantic understanding model;

[0062] Step S3: Model pre-training: using a masked language model to pre-train the semantic understanding model, and setting up paragraph aggregation prediction simultaneously;

[0063] Step S4: Model adjustment: using the literature data set to train and adjust the semantic understanding model to obtain an adjusted semantic understanding model;

[0064] Step S5: semantic analysis, using the adjusted semantic understanding model to perform semantic analysis and generate a deep semantic representation of the text.

[0065] Embodiment 3, based on the above embodiment, step S2 specifically includes the following contents:

[0066] Input embedding layer: The RoBERTa model is selected as the basic model, and the literature dataset is tokenized. The text data in the literature dataset is split into subword fragments using subword segmentation. The subword fragments are converted into vocabulary index format and passed into the RoBERTa model. The deep semantic features of the literature dataset are extracted through a multi-layer Transformer architecture to obtain a chapter-level representation.

[0067] Encoding layer: Based on the Transformer architecture, it includes 24 self-attention mechanism layers and a feedforward neural network

[0068] Hierarchical aggregation layer: The text is processed in a hierarchical manner, and the text data is divided into three levels: sentence, paragraph, and chapter. The sentences are encoded using Bi-LSTM, and the semantic representation of each sentence is extracted. The sentence representations are aggregated at the paragraph level, and the weight distribution between sentences is calculated through the self-attention mechanism to generate the global representation of the paragraph. At the chapter level, the paragraph representation is further aggregated and optimized through the hierarchical attention mechanism to generate the semantic representation of the entire text.

[0069] Knowledge graph fusion layer: Based on the external knowledge graph, the RotaE model is used to build entity embeddings. The fusion strategy of the attention mechanism is used to weightedly fuse entity embeddings with deep semantic features to generate semantic representations with background information.

[0070] Output layer: Receives the semantic representations from the hierarchical aggregation layer and the knowledge graph fusion layer, and maps them into vector outputs.

[0071] In this embodiment, the encoding layer includes 24 self-attention mechanism layers and a feedforward neural network to further encode and process the contextual information in the literary work data. Through the self-attention mechanism, the system assigns weights according to the importance of each word in the entire text, and fully understands the association and semantic connection between sentences.

[0072] The sentence level in the hierarchical aggregation layer uses Bi-LSTM to encode each sentence, extract the semantic representation of each sentence, capture the sequential dependency information within the sentence, and retain the bidirectional semantic features of the sentence. The paragraph level aggregates the representation of the sentences in each paragraph through the self-attention mechanism, calculates the weight distribution between sentences, determines the importance of the sentences, and thus generates the global semantic representation of the paragraph. At the chapter level, the semantic representation of the paragraph is further aggregated through the hierarchical attention mechanism, and the importance of each paragraph is weighted and optimized to generate the semantic representation of the entire text, ensuring that the system effectively captures the important information and core themes in the text;

[0073] The fusion strategy of the knowledge graph fusion layer uses the fusion strategy of the attention mechanism to weightedly fuse the generated entity embedding with the deep semantic features extracted by RoBERTa.

[0074] This embodiment adopts the RoBERTa model, combined with hierarchical aggregation, multi-level information fusion and knowledge graph to perform semantic analysis on the literary work dataset. The RoBERTa model is used to perform deep pre-training on text data to capture the complex semantic relationships in literary works. Combined with hierarchical aggregation, the content of long literary works is understood in a hierarchical manner, thereby avoiding information loss or one-sided analysis. The knowledge graph is used to improve the ability to understand the relationship between entities in characters, events and timelines in literary works.

[0075] Embodiment 4, based on the above embodiment, step S3 specifically includes the following steps:

[0076] Step S31: masking language model setting, select 15% of the words in the literature data set to mask and replace them with MASK. The goal of the masking language model is to predict the masked words;

[0077] Step S32: Setting paragraph aggregation tags: setting a target tag for each input paragraph through an external knowledge graph. These target tags are called paragraph aggregation tags.

[0078] Step S33: model input, inputting the masked literature dataset and paragraph aggregation labels into the masked language model;

[0079] Step S34: model training, using the cross entropy loss function to predict the masked words and paragraph aggregation labels, and combining the two prediction results into a comprehensive loss function;

[0080] Step S35: Model optimization, using the DTO-PSO hybrid optimization algorithm to optimize the masked language model and update the model parameters.

[0081] In this embodiment, the masking example is: for the input sentence "He looked at the beautiful sunset at dusk", the masking operation replaces "dusk" and "beautiful" with "MASK", generating the sentence "He looked at the MASK sunset at MASK";

[0082] The paragraph aggregation tag reflects the main semantic information and background knowledge in the paragraph. For example, for a paragraph containing sentences related to "war and peace", the paragraph aggregation tag is set to "historical background" or "war theme";

[0083] An input example of step S33 is: "He was watching the sunset while MASKing" and the paragraph aggregation tag "natural landscape" are input into the masking language model together.

[0084] The masked language model is used to pre-train the semantic understanding model, and the DTO-PSO hybrid optimization algorithm is used to optimize the masked language model. The information entropy feedback mechanism and constraint processing mechanism are introduced to improve the depth and accuracy of semantic understanding. The state of the particle swarm is evaluated by information entropy, and a balance is found between exploring new semantic structures and refining existing semantic understanding. In the global search process, the optimization results are refined through local search. Through the optimized masked language model and information entropy regulation, important information is extracted from a wider range of texts, and accurate summaries are generated in combination with semantic understanding.

[0085] Embodiment 5: This embodiment is based on the above embodiment. In step S35, the DTO-PSO hybrid optimization algorithm is used to train the masked language model, which specifically includes the following steps:

[0086] Step S351: Initialize the particle swarm, randomly generate the position of particles, each particle represents a parameter solution of the masking language model, introduce an adaptive weighting factor, define a feasible solution space, check whether the position of each particle is in the feasible solution space, and if it is not in the feasible solution space, use the repair mechanism to return to the feasible solution space. The feasible solution space is as follows: ;

[0087] in, is the feasible solution space, is the real number space, is a variable, is an inequality constraint, is an equality constraint, for The index of for The index of

[0088] Step S352: Introduce an information entropy feedback mechanism to evaluate the particle swarm through information entropy. The information entropy formula is: ;

[0089] in, For the During the round iteration, the information entropy of the particle swarm is is the total number of particles in the particle swarm, For the The particle in Probability distribution of fitness in iterations;

[0090] When the information entropy decreases and the particle swarm tends to concentrate, the exploration range of the search area is increased. When the information entropy is low, the particle speed is increased. The formula for updating the particle speed is: ;

[0091] in, It is The particle in The speed in the iteration, It is The particle in The speed in iterations;

[0092] Introduce the constraint processing mechanism, calculate the particle fitness, and introduce the penalty function into the original objective function. The formula used is as follows: ;

[0093] in, is the fitness function after constraint processing, is the original objective function, is an inequality constraint, is an equality constraint, and is the penalty coefficient;

[0094] Step S353: Use the PSO formula to update the particle position and velocity. The formula used is as follows: ; ;

[0095] in, is the inertia weight, , are individual learning factors and social learning factors, For the The particle in The optimal position of an individual in the iteration, For the The global optimal position in the iteration, For the The particle in The current position in iterations;

[0096] Constraint mechanism processing, after updating the position, check the particle Whether the constraints of the feasible solution space are met. If not, a penalty mechanism is applied to modify the fitness;

[0097] Step S354: Introduce DTO local search. After each particle update, use DTO to perform local optimization on the particle. The formula used is as follows: ;

[0098] in, It is The new position of a particle after local search, It is The particle in The initial position in the iteration, is the step size coefficient, The dimension is A random vector of ;

[0099] Constraint processing, DTO local search, check particles Whether the constraints of the feasible solution space are met. If not, a penalty mechanism is applied to modify the fitness;

[0100] Step S355: Global and local optimal update, the formula used is as follows: ; ;

[0101] in, It is The particle in The optimal position of the new individual after iterations is: is the fitness function, For the The global optimal position after iterations;

[0102] Step S356: Output the result. Output, as the optimal parameter solution for the masked language model.

[0103] In this embodiment, the initial positions of 30 particles are randomly generated, each particle represents a parameter solution of the RoBERTa model, and the initial values ​​of the particle positions are randomly distributed in the range of [−1,1][-1,1][−1,1]. Assume that the constraint condition is that the parameters of the RoBERTa model must be kept between [−1,1][-1,1][−1,1], so any parameters outside this range will be corrected.

[0104] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0105] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0106] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. A semantic understanding and automatic summary generation system for literary works, characterized by: It includes text collection module, semantic understanding module, topic extraction module, summary generation module and optimization module, which specifically include the following contents: The text collection module collects text data, including various literary works, and performs data cleaning, word segmentation and part-of-speech tagging; The semantic understanding module constructs a data set of literary works, uses the RoBERTa model, combines hierarchical aggregation, multi-level information fusion and knowledge graph to perform semantic analysis on the data set of literary works, and obtains the deep meaning of literary works; The topic extraction module uses a clustering algorithm to extract topics and key concepts from literary works; The summary generation module automatically generates a summary of a literary work by combining the deep meaning, theme and key concepts of the literary work based on a sequence-to-sequence model; The optimization module collects user feedback, performs manual corrections, and continuously optimizes.

2. The system for semantic understanding and automatic summary generation of literary works according to claim 1, characterized in that: Semantic analysis is performed in the semantic understanding module, which specifically includes the following steps: Step S1: text data preparation, preprocessing the text data, including noise removal and text segmentation, obtaining the processed text data and organizing it into a literature data set; Step S2: Model construction: construct a model based on RoBERTa, add a hierarchical aggregation layer and a knowledge graph enhancement layer to obtain a semantic understanding model; Step S3: Model pre-training: using a masked language model to pre-train the semantic understanding model, and setting up paragraph aggregation prediction simultaneously; Step S4: Model adjustment: using the literature data set to train and adjust the semantic understanding model to obtain an adjusted semantic understanding model; Step S5: semantic analysis, using the adjusted semantic understanding model to perform semantic analysis and generate a deep semantic representation of the text.

3. A system for semantic understanding and automatic summary generation of literary works according to claim 2, characterized in that: Step S2 specifically includes the following contents: Input embedding layer: The RoBERTa model is selected as the basic model, and the literature dataset is tokenized. The text data in the literature dataset is split into subword fragments using subword segmentation. The subword fragments are converted into vocabulary index format and passed into the RoBERTa model. The deep semantic features of the literature dataset are extracted through a multi-layer Transformer architecture to obtain a chapter-level representation. Encoding layer: Based on the Transformer architecture, it includes 24 self-attention mechanism layers and a feedforward neural network Hierarchical aggregation layer: The text is processed in a hierarchical manner, and the text data is divided into three levels: sentence, paragraph, and chapter. The sentences are encoded using Bi-LSTM, and the semantic representation of each sentence is extracted. The sentence representations are aggregated at the paragraph level, and the weight distribution between sentences is calculated through the self-attention mechanism to generate the global representation of the paragraph. At the chapter level, the paragraph representation is further aggregated and optimized through the hierarchical attention mechanism to generate the semantic representation of the entire text. Knowledge graph fusion layer: Based on the external knowledge graph, the RotaE model is used to build entity embeddings. The fusion strategy of the attention mechanism is used to weightedly fuse entity embeddings with deep semantic features to generate semantic representations with background information. Output layer: Receives the semantic representations from the hierarchical aggregation layer and the knowledge graph fusion layer, and maps them into vector outputs.

4. The system for semantic understanding and automatic summary generation of literary works according to claim 3, characterized in that: Step S3 specifically includes the following steps: Step S31: masking language model setting, select 15% of the words in the literature data set to mask and replace them with MASK. The goal of the masking language model is to predict the masked words; Step S32: Setting paragraph aggregation tags: setting a target tag for each input paragraph through an external knowledge graph. These target tags are called paragraph aggregation tags. Step S33: model input, inputting the masked literature dataset and paragraph aggregation labels into the masked language model; Step S34: model training, using the cross entropy loss function to predict the masked words and paragraph aggregation labels, and combining the two prediction results into a comprehensive loss function; Step S35: Model optimization, using the DTO-PSO hybrid optimization algorithm to optimize the masked language model and update the model parameters.

Images