Multilingual news abstract extraction method based on reinforcement subgraph mining

Abstract

The application relates to a multilingual news abstract extraction method based on reinforcement subgraph mining, and belongs to the technical field of natural language processing and artificial intelligence. The application is proposed in view of the problems of the traditional extractive abstract method, such as insufficient coherence between sentences, much redundant information, low abstract quality and the like in long document processing, and comprises the following steps: document multi-level semantic heterogeneous graph construction and initialization; feature enhancement is performed on the sentence nodes through a heterogeneous graph hierarchical structure encoder, a global context encoder and an extraction history encoder; key subgraph mining is strengthened; the abstract extraction process is modeled as a multi-step Markov decision process, key sentence nodes are dynamically selected based on a sentence-level feature graph to form an abstract subgraph, and the selection process is optimized through a reinforcement learning strategy. The application is suitable for automatic abstract generation tasks of long texts such as scientific literature, academic papers and long news reports, and has good interpretability, adaptability and abstract quality.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing and artificial intelligence technology, specifically relating to a multilingual news summary extraction method based on reinforced subgraph mining. Background Technology

[0002] With the rapid development of the information age, digital content has experienced explosive growth, among which long documents, such as academic papers, technical reports, lengthy news articles, and policy documents, are playing an increasingly important role in information dissemination and knowledge accumulation. These documents are typically characterized by their length, complex structure, high information density, and multi-layered interweaving of themes, making it difficult for users to grasp their core ideas and key information in a short time. Therefore, how to efficiently, accurately, and automatically extract concise, coherent, and complete summaries from massive amounts of long text has become a crucial research task with wide-ranging applications in the field of natural language processing.

[0003] Extractive summarization, as one of the mainstream techniques for text summarization, directly selects representative sentences or fragments from the original text and combines them in a certain order to form a summary. Compared with generative summarization methods, extractive methods have three significant advantages: First, they offer strong interpretability, as the summary content comes directly from the original text, avoiding factual errors or "illusions" that may arise from generated text; second, they offer high accuracy, preserving the true information and key facts of the original text to the greatest extent possible; and third, the technology is relatively mature, requiring relatively low computational power, making it suitable for scenarios with stringent reliability requirements, such as legal document summarization, medical report summarization, and scientific literature reviews. Therefore, in the field of long document processing, extractive summarization remains a key method of focus and adoption in both industry and academia.

[0004] However, despite the significant value of extractive summarization in practical applications, traditional and existing mainstream methods still face several core challenges that urgently need to be addressed:

[0005] First, the lack of coherence in the summary is a prominent issue. Traditional methods, such as those based on word frequency statistics and graph ranking algorithms (like TextRank and its improved versions), mostly employ a strategy of independently scoring sentences and then greedily selecting sentences based on those scores. While this strategy can select sentences with high local importance, it severely neglects the semantic connections, logical progressions, and textual structure information between sentences. The selected set of sentences often lacks natural transitions and connections, resulting in a stiff, disjointed, and poorly coherent summary that hinders the smooth delivery of information and the user's comprehension experience.

[0006] Secondly, information redundancy is prevalent. Long documents often contain repeated elaborations on the same topic from different angles or in different chapters. Traditional methods lack effective measurement and constraints on the semantic repetition between selected content and candidate sentences when selecting sentences, easily leading to the repeated selection of semantically similar sentences, resulting in repetitive summary content and reduced information density. This not only wastes limited summary length quotas but also reduces the conciseness and information value of the summary.

[0007] Secondly, there is a lack of adaptability to document structure and length. Documents from different fields and of different types (such as research papers and news reports) vary greatly in structure, length, language style, and information distribution. Traditional methods usually rely on a fixed abstract compression rate (such as retaining 20% ​​of the original text) or a static feature combination strategy, making it difficult to dynamically adjust according to the specific document's internal structure and the distribution of information importance. This results in abstracts that may omit key information or include too many important details in some documents with special structures, leading to inconsistent abstract quality.

[0008] To address these challenges, researchers have recently explored more advanced model architectures to improve the performance of extractive summarization. The rise of Graph Neural Networks (GNNs) has provided a new approach to document modeling. By treating each sentence in a document as a node in a graph and constructing edges based on semantic similarity, adjacency, and referential relationships between sentences, a sentence graph is formed. Utilizing the message passing mechanism of GNNs, global information can be effectively aggregated, capturing complex dependencies between sentences and the overall structure of the document. This offers the potential to improve the coherence and information completeness of summaries.

[0009] Meanwhile, the success of reinforcement learning (RL) in sequential decision-making problems provides a natural framework for the multi-step decision-making process of summary generation. By modeling the summary generation process as a sequential decision-making task in which an agent selects sentences in order, and designing reasonable reward functions (such as measuring summary quality based on metrics like Rouge, penalizing redundancy, and encouraging the coverage of key information), reinforcement learning can optimize sentence selection strategies in an end-to-end manner, learning globally optimal decisions oriented towards the final summary quality. Existing studies such as MemSum and GoSum have successfully introduced reinforcement learning into extractive summarization, achieving performance superior to traditional methods.

[0010] However, current extractive summarization methods incorporating reinforcement learning still have significant limitations. Firstly, these methods often fail to fully and explicitly utilize the graph structure information of the document during the decision-making process. The graph structural relationships between sentences contain rich semantic connections and discourse constraints, but existing methods mostly treat sentences as independent states or only implicitly learn relationships through the encoder, failing to use the graph structure as an explicit constraint in the decision-making process. This results in the generated summary being structurally disconnected from the discourse logic of the original text. Secondly, in the process of identifying and extracting key information subsets (i.e., sentence subgraphs constituting the summary) from the document graph, existing methods lack dynamic and flexible mining mechanisms. They typically employ predefined or heuristic subgraph generation patterns, unable to adaptively adjust the direction and scope of subsequent exploration based on the state of the currently selected summary and the overall topology of the document graph, thus limiting the model's ability to handle complex and diverse documents.

[0011] Therefore, to address the shortcomings of existing technologies in the aforementioned aspects, there is an urgent need for a novel extractive summarization method that can deeply integrate document graph structure information and utilize reinforcement learning to achieve dynamic and adaptive key subgraph mining. This method should be able to fully consider the structural relationship between the currently selected summary subgraph and the full document graph in each decision step of the reinforcement learning agent. It should encode global structural information through graph neural networks and design a corresponding reward mechanism and action space to guide the agent to dynamically mine key sentence subgraphs that are comprehensive in information coverage, semantically coherent, and have low redundancy, thereby significantly improving the overall quality, coherence, and adaptability of long document summarization. Summary of the Invention

[0012] The technical problem this invention aims to solve is: This invention provides a multilingual news summary extraction method based on reinforcement subgraph mining, which aims to address the problems of insufficient coherence, high redundancy, and poor adaptability in existing long document extraction summarization methods. The core of this method lies in modeling documents as multi-level semantic heterogeneous graphs. Through the deep integration of graph neural networks and reinforcement learning, the summary generation process is transformed into a multi-step, adaptive key subgraph dynamic mining task on the document graph. This method can effectively capture the semantic relationships between sentences and the text structure, significantly improving the coherence, conciseness, and adaptability to diverse documents while ensuring comprehensive information coverage.

[0013] To achieve the above objectives, the present invention adopts the following technical solution: a multilingual news summary extraction method based on enhanced subgraph mining, comprising the following steps:

[0014] Step 1: Construction and initialization of multi-level semantic heterogeneous graph of document: Construct the input document into a heterogeneous graph containing three types of nodes: words, sentences, and paragraphs, and initialize the representation of each node;

[0015] Step 2, Sentence-level feature encoding: Feature enhancement of sentence nodes is performed through heterogeneous graph hierarchical encoder, global context encoder and extract history encoder;

[0016] Step 3, Enhance Key Subgraph Mining: Model the summary extraction process as a multi-step Markov decision process, dynamically select key sentence nodes based on sentence-level feature maps to form summary subgraphs, and optimize the selection process through reinforcement learning strategies.

[0017] Further, in step 1, the heterogeneous graph construction includes:

[0018] Node set = ∪ ∪ ,in, For word nodes, For sentence nodes, Paragraph node;

[0019] edge set = ∪ ∪ ∪ , , , , These represent the edges from a word to its sentence, from a sentence to a word it contains, from a word to its paragraph, and from a paragraph to a sentence it contains.

[0020] Furthermore, the node initialization method in step 1 is as follows:

[0021] Word nodes are initialized using a pre-trained word vector model;

[0022] Sentence nodes and paragraph nodes obtain their initial representations by encoding and pooling the word embeddings they contain:

[0023]

[0024]

[0025] in, For words in the sentence, , For words in the paragraph, Enc represents a bidirectional LSTM encoder, and MHPool represents a multi-head pooling layer; , These are the initial representations of the word embeddings contained in sentence nodes and paragraph nodes, respectively.

[0026] Furthermore, in step 2, the heterogeneous graph hierarchical encoder uses a graph attention network for hierarchical message passing, specifically including:

[0027] Paragraph node update: Update the paragraph representation based on the information of adjacent sentence nodes;

[0028]

[0029]

[0030] Sentence node update: Enhance sentence representation based on updated paragraph node information:

[0031]

[0032] GAT stands for Graph Attention Network, and FFN stands for Feedforward Network. This represents the attention pooling result from sentence to paragraph, which is the result of the information aggregation process. It shows the process by which the model uses the GAT mechanism to pass information from adjacent sentence nodes s to paragraph nodes p. This is the initialization representation of the paragraph node. This is the initialization representation of the sentence node. This is the updated paragraph node representation. This represents the updated sentence node representation, which is the current sentence feature.

[0033] Furthermore, in step 2, the global context encoder uses a bidirectional LSTM to encode all sentence nodes to capture the overall context information of the document. The specific operation is as follows:

[0034]

[0035] in, This represents the sentence representation after global context enhancement, also known as global context features. Representation layer normalization, ) represents a bidirectional long short-term memory network. This represents the updated sentence node representation of the input, which is the current sentence feature.

[0036] Furthermore, in step 2, the extraction history encoder employs a multi-head attention mechanism to encode the extracted sentences and their adjacent candidate sentences. The specific operation is as follows:

[0037]

[0038]

[0039]

[0040] in, This represents the candidate sentences. For the extracted sentence representation, MHA is a multi-head attention mechanism. Representation layer normalization, This represents a feedforward neural network. This represents the feature input for the next layer, which is the final historical embedding and also the output feature vector of the (h+1)th layer in the historical encoder.

[0041] Furthermore, in step 3, the enhanced key subgraph mining specifically includes:

[0042] State representation: The current sentence features, global context features, and extracted historical features are concatenated to form the state. :

[0043] =[ ; ]

[0044] in, This represents the updated sentence node representation, which is the current sentence feature. This represents the sentence representation after global context enhancement, also known as global context features. This indicates that the output feature vector of the h-th layer in the historical encoder is extracted, which is to say, historical features are extracted.

[0045] Action space: defined as the set of adjacent sentence nodes of the currently extracted sentence.

[0046]

[0047] in, This represents the set of adjacent sentence nodes of the currently extracted sentence. (·) denotes the neighbor relation function, used to define the topological structure of the connection relationships between sentences. It is a set of selected sentences;

[0048] Reward function: Calculates immediate reward based on the Rouge scores of the generated and reference summaries.

[0049]

[0050] in, This represents the immediate reward value, which is the total reward value that the environment gives to the agent after it completes a summary extraction. Represents the Rouge-1 fraction. Represents Rouge-2 score, Represents Rouge-L score;

[0051] Policy Network: Outputs stopping probability and candidate sentence scores, dynamically deciding whether to continue extraction.

[0052]

[0053]

[0054] in, The policy function represents the state. Next, take action The probability distribution; Represents the stopping probability in state The model then determines the probability of terminating the extraction process. Represents the probability of sentence selection in state. Given the condition "whether to stop", select the sentence. The conditional probability; and Represents the scoring function; This indicates that the logical condition is to stop extraction, and the stop flag is true, meaning the stop condition is met. This indicates that the logical condition is to continue extraction, and the stop flag is false, meaning the stop condition is not met. Indicates model parameters, Sentences indicating choices.

[0055] Furthermore, the policy network is optimized using the policy gradient method, with the objective function being to maximize the cumulative reward:

[0056]

[0057] in, Indicates the state Select action The probability, where T represents the trajectory length. This represents the immediate reward value, which is the total reward value that the agent receives from the environment after completing a digest extraction. This represents the expectation under the policy distribution, also known as the mathematical expectation operator. This represents the objective function or expected value. This represents the gradient of the objective function.

[0058] The beneficial effects of this invention are:

[0059] This invention proposes a multilingual news summarization method based on reinforcement subgraph mining. By modeling documents as multi-level semantic graphs and integrating reinforcement learning for dynamic subgraph mining, the method significantly improves the quality and adaptability of long document summarization. Specifically, this invention has the following beneficial effects:

[0060] 1. Significantly Improves the Coherence and Information Completeness of Summaries: By constructing a heterogeneous graph containing three layers of nodes—words, sentences, and paragraphs—and utilizing a graph attention network for hierarchical information transmission, this method can explicitly model the semantic connections, positional proximity relationships, and paragraph-level thematic consistency between sentences. This structured representation allows the model to fully consider the context and discourse logic when selecting sentences, thereby generating summaries with natural semantic coherence and logical flow. Compared to traditional methods that select sentences independently, the summaries generated by this method show significant improvements in reading fluency and overall coherence, and are more in line with human reading habits.

[0061] 2. Effectively Reduces Summary Redundancy and Achieves Adaptive Length Control: The method innovatively models the summary generation process as a multi-step reinforcement learning task and designs a dedicated history extraction encoder and reward mechanism. In each decision step, the model evaluates the semantic similarity between candidate sentences and the selected summary through a multi-head attention mechanism, effectively suppressing the selection of repetitive information. Simultaneously, the introduced dynamic stopping mechanism allows the policy network to adaptively predict the termination point based on the document content, overcoming the limitations of a fixed compression rate. This enables the generation of concise summaries with appropriate length and high information density while ensuring no loss of core information, effectively solving the problems of content redundancy and rigid length in traditional methods.

[0062] 3. Superior performance on multiple standard datasets: Systematic experiments on classic long document summarization datasets such as CNN / DailyMail, PubMed, and arXiv demonstrate that this method significantly outperforms traditional extractive baseline models (such as TextRank and SummaRuNNer) and models combining simple graph structures on automatic evaluation metrics such as Rouge and BERTScore. Furthermore, compared to some advanced generative summarization models, this method shows greater advantages in factual consistency and content accuracy, proving its effectiveness and competitiveness in practical applications.

[0063] 4. Excellent interpretability and broad applicability: Due to the extraction framework, each sentence in the summary is directly derived from the original text, making the summary generation process transparent and easy to trace and verify. Furthermore, the decision-making process based on graph structure and reinforcement learning allows for intuitive analysis of which sentences the model focused on and their relationships. These characteristics make this method suitable not only for scenarios with high requirements for factual accuracy (such as scientific literature, legal documents, and medical report summaries), but also easy to integrate into practical systems such as news aggregation and knowledge management, providing an efficient and reliable technical solution for handling diverse and large-scale long document summarization tasks. Attached Figure Description

[0064] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0065] Figure 1 This is a schematic diagram of the overall architecture of the model of the present invention;

[0066] Figure 2 This is a flowchart of the method of the present invention;

[0067] Figure 3 This is a schematic diagram of message passing in a heterogeneous graph hierarchy;

[0068] Figure 4 A comparison chart showing the impact of different graph types on summarization performance;

[0069] Figure 5 A comparison chart of model convergence speeds;

[0070] Figure 6 This is a flowchart of the method of the present invention. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Therefore, the following detailed description of the embodiments of the present invention is not intended to limit the scope of the claimed invention, but merely represents some embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. It should be noted that, in the absence of conflict, the embodiments and features and technical solutions in the embodiments of the present invention can be combined with each other. It should be noted that similar reference numerals and letters in the following drawings indicate similar items; therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0072] To address issues such as information redundancy, loose structure, and semantic dispersion in long document summaries, this embodiment provides a multilingual news summary extraction method based on reinforcement subgraph mining. This method aims to improve the information integrity and semantic coherence of summaries by constructing multi-level semantic heterogeneous graphs and dynamically mining key subgraphs using reinforcement learning. This approach is suitable for automatic summarization of long texts such as scientific literature, technical reports, and long news articles, and provides a new technical path for summary modeling in low-resource scenarios.

[0073] Specifically, such as Figures 1-6 As shown, this multilingual news summary extraction method based on reinforcement subgraph mining includes the following steps:

[0074] Step 1: Construction and initialization of multi-level semantic heterogeneous graph of document: Construct the input document into a heterogeneous graph containing three types of nodes: words, sentences, and paragraphs, and initialize the representation of each node;

[0075] Step 1 includes:

[0076] Step 1.1: First, perform structured parsing on the input long document; specifically, model the document's words, sentences, and paragraphs as three different types of nodes in a heterogeneous graph.

[0077] The heterogeneous graph construction includes:

[0078] Node set = ∪ ∪ ,in, For word nodes, For sentence nodes, Paragraph node;

[0079] edge set = ∪ ∪ ∪ , , , , These represent the edges from a word to its sentence, from a sentence to a word it contains, from a word to its paragraph, and from a paragraph to a sentence it contains.

[0080] Step 1.2: Initialize the representation of the three types of nodes. The node initialization method in Step 1 is as follows:

[0081] Word nodes: Initialized using a pre-trained word vector model;

[0082] Specifically, initialization is performed using pre-trained word vector models (such as Word2Vec, GloVe, or BERT's token embeddings); for each word... Its initial vector is represented as ;

[0083] Sentence nodes and paragraph nodes obtain their initial representations by encoding and pooling the word embeddings they contain:

[0084]

[0085]

[0086] in, For words in the sentence, , For words in the paragraph, Enc represents a bidirectional LSTM encoder, and MHPool represents a multi-head pooling layer; , These are the initial representations of the word embeddings contained in sentence nodes and paragraph nodes, respectively.

[0087] Furthermore, specifically, it can include:

[0088] Sentence node: The initial representation of a sentence node s It is obtained by aggregating the embeddings of all the words it contains. For example, using mean pooling:

[0089]

[0090] Paragraph Node: Similarly, the initialization representation of a paragraph node p We obtain the following by aggregating the initial representations of all the sentences it contains:

[0091]

[0092] Finally, edges are constructed between nodes based on the document's natural structure: word-sentence membership edges (a word belongs to its sentence), sentence-paragraph membership edges (a sentence belongs to its paragraph), and sentence-sentence adjacency edges (constructed based on the sentence's positional adjacency or semantic similarity within the document); thus, a document heterogeneous graph containing rich hierarchical and relational information is formed. ,in For a set of nodes, Let it be the set of edges;

[0093] Step 2, Sentence-level feature encoding: Feature enhancement of sentence nodes is performed through heterogeneous graph hierarchical encoder, global context encoder and extract history encoder;

[0094] This step aims to enhance the feature representation of each sentence node by encoding structural information of the document graph, global context, and selected summary history, thus providing high-quality feature input for subsequent key subgraph mining;

[0095] Furthermore, in step 2, the heterogeneous graph hierarchical encoder uses a multi-layer graph attention network (GAT) for hierarchical message passing. Specifically, information is aggregated from word nodes to their respective sentence nodes, then from sentence nodes to their respective paragraph nodes, and finally from paragraph nodes back to sentence nodes, so that each sentence node can integrate its local vocabulary information with the topic information of its respective paragraph.

[0096] Specifically, it includes:

[0097] Paragraph node update: Update the paragraph representation based on the information of adjacent sentence nodes;

[0098]

[0099]

[0100] Sentence node update: Enhance sentence representation based on updated paragraph node information:

[0101]

[0102] GAT stands for Graph Attention Network, and FFN stands for Feedforward Network. This represents the attention pooling result from sentence to paragraph, which is the result of the information aggregation process. It shows the process by which the model uses the GAT mechanism to pass information from adjacent sentence nodes s to paragraph nodes p. This is the initialization representation of the paragraph node. This is the initialization representation of the sentence node. This is the updated paragraph node representation. This represents the updated sentence node representation, which is the current sentence feature.

[0103] Furthermore, in step 2, the global context encoder uses a bidirectional LSTM to encode all sentence nodes to capture the overall context information of the document. The specific operation is as follows:

[0104]

[0105] in, This represents the sentence representation after global context enhancement, also known as global context features. Representation layer normalization, ) represents a bidirectional long short-term memory network. This represents the updated sentence node representation of the input, which is the current sentence feature.

[0106] Furthermore, in step 2, the extraction history encoder employs a multi-head attention mechanism to encode the extracted sentences and their adjacent candidate sentences. The specific operation is as follows:

[0107]

[0108]

[0109]

[0110] in, This represents the candidate sentences. For the extracted sentence representation, MHA is a multi-head attention mechanism. Representation layer normalization, This represents a feedforward neural network. This represents the feature input for the next layer, which is the final historical embedding and also the output feature vector of the (h+1)th layer in the historical encoder.

[0111] Step 3, Enhance Key Subgraph Mining: Model the summary extraction process as a multi-step Markov decision process, dynamically select key sentence nodes based on sentence-level feature maps to form summary subgraphs, and optimize the selection process through reinforcement learning strategies.

[0112] Furthermore, in step 3, the enhanced key subgraph mining specifically includes:

[0113] State representation: The current sentence features, global context features, and extracted historical features are concatenated to form the state. :

[0114] =[ ; ]

[0115] in, This represents the updated sentence node representation, which is the current sentence feature. This represents the sentence representation after global context enhancement, also known as global context features. This indicates that the output feature vector of the h-th layer in the historical encoder is extracted, which is to say, historical features are extracted.

[0116] Action space: defined as the set of adjacent sentence nodes of the currently extracted sentence.

[0117]

[0118] in, This represents the set of adjacent sentence nodes of the currently extracted sentence. (·) denotes the neighbor relation function, used to define the topological structure of the connection relationships between sentences. It is a set of selected sentences;

[0119] Reward function: Calculates immediate reward based on the Rouge scores of the generated and reference summaries.

[0120]

[0121] in, This represents the immediate reward value, which is the total reward value that the environment gives to the agent after it completes a summary extraction. Represents the Rouge-1 fraction. Represents Rouge-2 score, Represents Rouge-L score;

[0122] Policy Network: Outputs stopping probability and candidate sentence scores, dynamically deciding whether to continue extraction.

[0123]

[0124]

[0125] in, The policy function represents the state. Next, take action The probability distribution; Represents the stopping probability in state The model then determines the probability of terminating the extraction process. Represents the probability of sentence selection in state. Given the condition "whether to stop", select the sentence. The conditional probability; and Represents the scoring function; This indicates that the logical condition is to stop extraction, and the stop flag is true, meaning the stop condition is met. This indicates that the logical condition is to continue extraction, and the stop flag is false, meaning the stop condition is not met. Indicates model parameters, Sentences indicating choices.

[0126] Furthermore, the policy network is optimized using the policy gradient method, with the objective function being to maximize the cumulative reward:

[0127]

[0128] in, Indicates the state Select action The probability, where T represents the trajectory length. This represents the immediate reward value, which is the total reward value that the agent receives from the environment after completing a digest extraction. This represents the expectation under the policy distribution, also known as the mathematical expectation operator. This represents the objective function or expected value. This represents the gradient of the objective function.

[0129] The purpose of the training and evaluation in this invention is to verify the effectiveness of the method through experiments and to conduct performance evaluation and comparative analysis on public datasets.

[0130] The implementation process is as follows:

[0131] First, training was performed on two long document summarization datasets, PubMed and arXiv. The Adam optimizer was used, and training was performed for 20 epochs on an NVIDIA RTX 3090 GPU. The batch size was set to 16, and the learning rate was 1×10⁻⁶. Gradient clipping can be used to avoid gradient explosion.

[0132] The method is trained and evaluated on scientific paper abstract datasets such as PubMed and arXiv. The abstract length is adaptively determined through a dynamic stopping mechanism, and the stopping threshold is set according to the characteristics of the dataset.

[0133] PubMed dataset:

[0134] arXiv dataset:

[0135] in, It is a preset numerical standard used to determine when the model stops extracting new sentences during the process of generating summaries;

[0136] Then, Rouge-1, Rouge-2, and Rouge-L were used as automatic evaluation indicators, while human evaluation was also conducted to score the generated abstracts from three dimensions: information completeness, fluency, and conciseness.

[0137] The experimental results are shown in Table 1 below:

[0138] Table 1 shows the comparative experimental results of the baseline model on the PubMed and arXiv datasets.

[0139] Analysis of experimental results:

[0140] 1. On PubMed and arXiv datasets, our method significantly outperforms baseline models such as Lead-10, MemSum, and GoSum on Rouge-1, Rouge-2, and Rouge-L metrics, and also outperforms large models such as GPT-3.5 / 4.

[0141] 2. Ablation experiments show, as shown in Table 2, that the heterogeneous graph encoder and the enhanced subgraph generation module contribute the most to performance, verifying the effectiveness of the multi-level semantic modeling and dynamic selection mechanism in this method.

[0142] Table 2 shows the ablation experiment results on the PubMed dataset.

[0143] 3. The results of manual evaluation are shown in Table 3. The summaries generated by this method are significantly better than the baseline model in terms of information completeness and conciseness, and the fluency is also at an acceptable level.

[0144] Table 3 shows the results of human evaluations on the PubMed and arXiv datasets.

[0145] Based on the above steps, the multilingual news summarization method based on reinforcement subgraph mining proposed in this embodiment effectively improves the information density and semantic coherence of long document summaries through multi-level semantic graph modeling and reinforcement learning dynamic mining mechanisms. This method exhibits good applicability and scalability in both unsupervised and weakly supervised scenarios, providing a new solution for automated long text summarization tasks.

[0146] Figure 3 This is a schematic diagram of heterogeneous graph hierarchical message passing, which explains in detail the internal working mechanism of the "heterogeneous graph hierarchical encoder". Passing Mechanism: Direction: Information is passed from the sentence level (sentences S1, S2) to the paragraph level (paragraphs P1, P2), interacts between paragraphs (paragraphs P1, P2, P3), and then feeds back to the sentence level (sentences S1, S2). Purpose: Through this hierarchical graph attention network (GAT) message passing, sentence nodes can capture the global hierarchical structure information of the document, thereby generating richer sentence embedding representations.

[0147] Figure 4This is a comparison chart of the impact of different graph types on summarizing performance. It shows the influence of different graph structures (similarity graph, topic graph, discourse graph) on the model's summarizing performance on the PubMed test set. Evaluation metrics include Rouge-1, Rouge-2, and Rouge-L. Discourse graph performed best: among all evaluation metrics, discourse graph performed the best. This demonstrates that graph structures containing rich logical relationships (such as topic, semantics, and discourse coherence) can help the model generate more coherent and accurate summaries. Similarity graph performed worst: similarity graph ranked lowest. This verifies the research hypothesis that simply selecting based on the similarity between nodes (sentences) can easily lead to duplicate (redundant) summary content, because similar sentences often express the same information.

[0148] Figure 5 This is a convergence speed analysis graph, comparing the convergence speed of the model corresponding to the method proposed in this invention with the baseline model during the training process. Compared to the baseline model, the model corresponding to the method proposed in this invention shows a significant convergence speed advantage in the early stages of training. The model can reach a stable performance level with fewer iterations. This advantage proves the effectiveness of "constraining through sentence relationship graphs". Utilizing the graph structure to limit the search space greatly reduces the model's search in invalid regions, thereby significantly improving training efficiency and performance.

[0149] The above examples are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above examples. Any modifications, alterations, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A multilingual news summary extraction method based on reinforced subgraph mining, characterized in that, Includes the following steps: Step 1: Construction and initialization of multi-level semantic heterogeneous graph of document: Construct the input document into a heterogeneous graph containing three types of nodes: words, sentences, and paragraphs, and initialize the representation of each node; Step 2, Sentence-level feature encoding: Feature enhancement of sentence nodes is performed through heterogeneous graph hierarchical encoder, global context encoder and extract history encoder; Step 3, Enhance Key Subgraph Mining: Model the summary extraction process as a multi-step Markov decision process, dynamically select key sentence nodes based on sentence-level feature maps to form summary subgraphs, and optimize the selection process through reinforcement learning strategies; In step 2, the heterogeneous graph hierarchical encoder uses a graph attention network for hierarchical message passing, specifically including: Paragraph node update: Update the paragraph representation based on the information of adjacent sentence nodes; ； ； Sentence node update: Enhance sentence representation based on updated paragraph node information: ； GAT stands for Graph Attention Network, and FFN stands for Feedforward Network. This represents the attention pooling result from sentence to paragraph, which is the result of the information aggregation process. It shows the process by which the model uses the GAT mechanism to pass information from adjacent sentence nodes s to paragraph nodes p. This is the initialization representation of the paragraph node. This is the initialization representation of the sentence node. This is the updated paragraph node representation. This represents the updated sentence node representation, which is the current sentence feature; In step 3, the enhanced key subgraph mining specifically includes: State representation: The current sentence features, global context features, and extracted historical features are concatenated to form the state. : =[ ; ]； in, This represents the updated sentence node representation, which is the current sentence feature. This represents the sentence representation after global context enhancement, also known as global context features. This indicates that the output feature vector of the h-th layer in the historical encoder is extracted, which is to say, historical features are extracted. Action space: defined as the set of adjacent sentence nodes of the currently extracted sentence. ； in, This represents the set of adjacent sentence nodes of the currently extracted sentence. (·) denotes the neighbor relation function, used to define the topological structure of the connection relationships between sentences. It is a set of selected sentences; Reward function: Calculates immediate reward based on the Rouge scores of the generated and reference summaries. ； in, This represents the immediate reward value, which is the total reward value that the environment gives to the agent after it completes a summary extraction. Represents the Rouge-1 fraction. Represents Rouge-2 score, Represents Rouge-L score; Policy Network: Outputs stopping probability and candidate sentence scores, dynamically deciding whether to continue extraction. ； ； in, The policy function represents the state. Next, take action The probability distribution; Represents the stopping probability in state The model then determines the probability of terminating the extraction process. Represents the probability of sentence selection in state. Given the condition "whether to stop", select the sentence. The conditional probability; and Represents the scoring function; This indicates that the logical condition is to stop extraction, and the stop flag is true, meaning the stop condition is met. This indicates that the logical condition is to continue extraction, and the stop flag is false, meaning the stop condition is not met. Indicates model parameters, Sentences indicating choices.

2. The multilingual news summary extraction method based on enhanced subgraph mining according to claim 1, characterized in that, In step 1, the heterogeneous graph construction includes: Node set = ∪ ∪ ,in, For word nodes, For sentence nodes, Paragraph node; edge set = ∪ ∪ ∪ , , , , These represent the edges from a word to its sentence, from a sentence to a word it contains, from a word to its paragraph, and from a paragraph to a sentence it contains.

3. The multilingual news summary extraction method based on enhanced subgraph mining according to claim 1 or 2, characterized in that, The node initialization method in step 1 is as follows: Word nodes are initialized using a pre-trained word vector model; Sentence nodes and paragraph nodes obtain their initial representations by encoding and pooling the word embeddings they contain: ； ； in, For words in the sentence, , For words in the paragraph, Enc represents a bidirectional LSTM encoder, and MHPool represents a multi-head pooling layer; , These are the initial representations of the word embeddings contained in sentence nodes and paragraph nodes, respectively.

4. The multilingual news summary extraction method based on enhanced subgraph mining according to claim 1, characterized in that, In step 2, the global context encoder uses a bidirectional LSTM to encode all sentence nodes to capture the overall context information of the document. The specific operation is as follows: ； in, This represents the sentence representation after global context enhancement, also known as global context features. Representation layer normalization, ) represents a bidirectional long short-term memory network. This represents the updated sentence node representation of the input, which is the current sentence feature.

5. The multilingual news summary extraction method based on enhanced subgraph mining according to claim 1, characterized in that, In step 2, the extraction history encoder uses a multi-head attention mechanism to encode the extracted sentence and its adjacent candidate sentences. The specific operation is as follows: ； ； ； in, This represents the candidate sentences. For the extracted sentence representation, MHA is a multi-head attention mechanism. Representation layer normalization, This represents a feedforward neural network. This represents the feature input for the next layer, which is the final historical embedding and also the output feature vector of the (h+1)th layer in the historical encoder.

6. The multilingual news summary extraction method based on enhanced subgraph mining according to claim 1, characterized in that, The policy network is optimized using the policy gradient method, with the objective function being to maximize the cumulative reward. ； in, Indicates the state Select action The probability, where T represents the trajectory length. This represents the immediate reward value, which is the total reward value that the agent receives from the environment after completing a digest extraction. This represents the expectation under the policy distribution, also known as the mathematical expectation operator. This represents the objective function or expected value. This represents the gradient of the objective function.

Images