An iterative text summarization system and method based on document topology.

Abstract

A system and method for iterative text summarization based on document topology, relating to the fields of artificial intelligence and natural language processing, addresses the problems of insufficient information extraction and redundancy control, inaccurate semantic expression, neglect of the global semantic structure within the document, and insufficient adaptability in low-resource scenarios in existing text summarization technologies. The system of this invention includes a text segmentation module, a generation model module, a context fusion module, and a final summary output module. After the input document is processed by the text segmentation module, the segmentation results are sent to the generation model module for initial summary generation. The context fusion module iteratively updates the initial summary output by the generation model module in each round. The final summary output module performs final correction on the summary output by the context fusion module. This method is not limited by document type, structure, or language, and has wide applicability.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and natural language processing, specifically to an iterative text summarization system and method based on document topology. Background Technology

[0002] Existing text summarization technologies are mainly divided into two categories: extractive summarization and abstractive summarization. Extractive summarization techniques directly select sentences or fragments as the final summary by identifying key information in the input document. The core of these methods lies in using inherent text features (such as word frequency, sentence position, and similarity) to score and rank the text, then constructing summaries based on the scores. Its main advantages are simplicity and high efficiency; however, because it relies solely on existing expressions in the text, it often suffers from redundant information, lack of coherence in the summary sentences, and insufficient deep semantic understanding. Abstractive summarization, on the other hand, focuses on using sequence-to-sequence (seq2seq) neural network models to encode the semantics of the original document and then generate new expressions to produce summaries that better conform to human language habits. However, in practical applications, abstractive summarization often faces problems such as semantic repetition, inaccuracy, and even factual errors. Therefore, in recent years, technologies based on the Transformer architecture and pre-trained models (such as PEGASUS, T5-PEGASUS, and MBART) have been widely applied to summarization tasks. These models significantly improve the accuracy and fluency of summarization by fully utilizing contextual semantic information and large-scale pre-training corpora, while demonstrating strong adaptability in long text summarization and cross-language summarization tasks. The closest existing technologies to this invention mainly focus on how to utilize the structural features within a document to improve summarization quality. For example, some studies divide documents into hierarchical structures or utilize a two-stage summarization strategy: the first stage extracts important segments from a global perspective, and the second stage uses refinement processing to generate a summary. These methods functionally capture both local and global semantic information in the document and can, to some extent, solve the summarization quality problems caused by information redundancy or insufficiency in a single summarization process.

[0003] However, existing technologies have the following defects and shortcomings in text summarization, and the main reasons and difficulties encountered in solving them can be summarized as follows:

[0004] (1) Insufficient information extraction and redundancy control;

[0005] Extractive summarization primarily relies on surface features such as word frequency, sentence position, and similarity to rank the "importance" of sentences in the original text, making it difficult to fully capture implicit semantic relationships. This results in the final summary often containing a lot of redundant information and lacking necessary generalization and coherence. Existing technologies, when determining sentence importance, fail to deeply understand the semantic hierarchy and structure of the text, easily including redundant or secondary information in the summary results.

[0006] Achieving accurate filtering of redundant information requires both a deep understanding of the complex semantic dependencies within the text and the ability of the model to dynamically distinguish between key information and noise. This presents significant challenges in both algorithm design and actual computation.

[0007] (2) Inaccuracy in generating semantic expressions;

[0008] Abstract summarization methods generate new text by performing deep semantic encoding on the original text, but in practice, semantic repetition, missing information, or errors are prone to occur. This is mainly because the model struggles to simultaneously ensure grammatical fluency and semantic accuracy when generating sequences, especially when dealing with long texts or loosely structured documents.

[0009] Generating text summaries with high accuracy and semantic coherence requires training and optimizing the model with a large amount of data. At the same time, it is necessary to introduce semantic error correction or semantic verification mechanisms at the algorithm level. Balancing generation efficiency and semantic accuracy while reducing error propagation is a challenge in current research.

[0010] (3) Ignoring the global semantic structure within the document;

[0011] Most existing technologies focus on extracting local information, while paying insufficient attention to the overall document structure and the semantic connections between its parts. For example, some methods fail to identify the inherent logical structure or thematic shifts between different paragraphs of a document, merely employing a "flattened" processing approach, resulting in generated summaries that fail to reflect the document's overall semantic coherence.

[0012] Capturing multi-layered and multi-dimensional semantic topology within a document requires designing an abstract method that can preserve local details while coordinating global information. This often necessitates more complex network structures and more efficient iterative fusion mechanisms. At the same time, it is easily affected by the length of the input text and semantic ambiguity, making it difficult to fundamentally solve the problem of global information loss.

[0013] (4) Insufficient adaptability in low-resource scenarios;

[0014] Current abstract summarization methods based on pre-trained models mostly rely on pre-training on large-scale data. When faced with scenarios where data is scarce or resources are limited in a specific domain, their generation performance often suffers significantly. Furthermore, the risk of model overfitting is high in these scenarios, resulting in insufficient robustness and generalizability of the generated summaries.

[0015] How to effectively utilize limited samples for model training under low-resource conditions, and ensure that the model can learn deep semantic information while avoiding overlearning of redundant information, is a major challenge that needs to be overcome in the technical implementation process.

[0016] In summary, existing technologies have made significant strides in improving the accuracy and coherence of text summarization systems, but they still suffer from the following shortcomings: 1. The capture of complex dependencies between semantic units within a document remains relatively weak, resulting in the generated summary not being entirely consistent with the overall semantics of the original text; 2. Long texts or documents with unclear structures are prone to losing key information or generating redundant summaries during the segmentation and information fusion process; 3. In low-resource scenarios, summarization methods based on large-scale pre-trained models often struggle to adequately adapt to situations with insufficient data. Solving these problems requires innovation in semantic extraction, information fusion, and model training strategies to reduce redundancy and erroneous information generation while ensuring the accuracy and coherence of summaries.

[0017] To address the aforementioned shortcomings, this invention proposes an iterative text summarization system and method based on document topology. By segmenting documents, modeling topological semantics, and fusion of hierarchical semantics, the system effectively improves the accuracy, coherence, and information density of summaries. Summary of the Invention

[0018] To address the problems of insufficient information extraction and redundancy control, inaccurate semantic expression, neglect of the global semantic structure within a document, and insufficient adaptability in low-resource scenarios in existing text summarization technologies, this invention provides an iterative text summarization system and method based on document topology.

[0019] An iterative text summarization generation system based on document topology structure, the system includes a text segmentation module, a generation model module, a context fusion module, and a final summary output module;

[0020] After the input document is processed by the text segmentation module, the segmentation results are sent to the generation model module for initial summary generation.

[0021] The context fusion module is used to receive the previous round of summary and the current semantic block, perform joint encoding and semantic fusion on the two, generate a context-enhanced representation, and transmit the context-enhanced representation to the generation model module.

[0022] The final summary output module performs a final correction on the summary output by the context fusion module.

[0023] This invention also provides an iterative text summarization method based on document topology, which is implemented using the aforementioned generation system. The implementation process of this method is as follows:

[0024] Step 1: The text segmentation module performs semantic segmentation and topology construction on the input document;

[0025] By using semantic similarity calculation, the input document is divided into several semantic blocks of independent semantic units, and the corresponding topological semantic network is constructed.

[0026] Step 2: Initial Summary Generation

[0027] For the first semantic block obtained in step one, an initial summary is generated using the generative model module. ;

[0028] Step 3: Iterative Fusion Update: For each subsequent semantic block, the summary generated in the previous round is jointly encoded with the current semantic block and then input into the context fusion module. The context fusion module extracts the key information in the current semantic block and fuses it with the global semantic information in the previous round summary to obtain the context-enhanced representation. Then, the generation model module generates an updated summary based on the context-enhanced representation until all semantic blocks are processed.

[0029] Step 4: Output the final summary through the final summary output module. .

[0030] The beneficial effects of this invention are:

[0031] The method of this invention utilizes document topology to segment text and iteratively generate summaries. This method exhibits significant innovation and advantages in several aspects, including segmentation strategy, semantic integration, contextual iterative updates, and low resource adaptability, setting a new benchmark for existing text summarization technologies and providing broad development prospects for various document information processing and patent technology applications.

[0032] This invention addresses the problems of excessive redundancy and insufficient compactness in existing text summarization techniques. It improves the semantic accuracy and overall consistency of summarization through an efficient semantic iterative fusion mechanism. It fully explores and utilizes the global semantic structure of documents, overcoming the shortcomings of existing methods that over-rely on local information. It improves the stability and robustness of model summarization under low-resource conditions. It has significant technological innovation and practical application prospects.

[0033] Specifically, it has the following advantages:

[0034] (1) Reduce redundant information and improve the compactness of the summary;

[0035] Existing technologies, in the process of abstract generation, rely solely on local features to select key information, which easily leads to redundant information in the abstract and fails to achieve efficient compression of the full text. The method of this invention aims to divide the document into multiple semantic blocks through analysis of the document's topological structure, and gradually fuse and refine information during the iterative process. This effectively reduces redundant information while preserving the core semantics, thus achieving a compact abstract.

[0036] (2) Improve the accuracy of semantic representation and generation;

[0037] Traditional abstract summarization techniques often suffer from semantic redundancy, information omissions, or generated errors during the generation process. This is because the models struggle to simultaneously capture global and local semantic relationships in long texts or loosely structured documents. This invention addresses this issue by employing a generative model combined with an iterative update mechanism. It fuses the contextual information of the previous-stage generated summary with the current text block, achieving a precise expression of the semantic details and overall structure of the entire text, thereby improving the accuracy and semantic consistency of the summary generation.

[0038] (3) Make full use of the document's global semantic structure;

[0039] Most existing technologies focus on local feature extraction, failing to capture the complex semantic dependencies between different parts of a document, resulting in an inability to fully reflect the document's global semantics when generating summaries. The method of this invention constructs a topological semantic network of the document by segmenting it into blocks based on semantic similarity and logical relationships. During iterative summarization, information from preceding and following blocks is continuously fused, ensuring that the final summary reflects both local details and maintains global semantic coherence.

[0040] (4) Improve summary generation performance in low-resource scenarios;

[0041] Current methods based on large-scale pre-trained models often face problems such as insufficient data and model overfitting in low-resource environments, resulting in weak robustness and generalization ability of summary generation. The method of this invention combines an iterative compression strategy and a contextual information filtering mechanism. By performing efficient information extraction and noise suppression within each semantic block, it can maintain high semantic capture ability and generation stability even under low-sample conditions, thereby improving the summary quality in low-resource scenarios.

[0042] (5) Construction and utilization of document topology;

[0043] This invention first divides a document into several semantic blocks through semantic similarity analysis, and then constructs a document topology based on the semantic dependencies between these blocks. This topology ensures that the different parts of the document are interconnected when generating a summary, guaranteeing that the summary covers the document's global semantics rather than relying solely on local information. Unlike other methods, this invention effectively identifies and processes the inherent logic and semantic transitions within each part of the document, thereby avoiding information omissions or duplications.

[0044] (6) Iterative summary generation and topology fusion;

[0045] This invention employs an iterative mechanism in the summary generation process, where each generated summary is based on the fusion of the previous round's output and the content of the current semantic block. Thus, as iterations proceed, semantic information from the preceding and following text is continuously integrated, ensuring that the generated summary maintains conciseness while preserving key information. This method significantly improves the coherence and accuracy of summaries, especially when dealing with structurally complex or long documents, guaranteeing the semantic integrity of the generated summary.

[0046] (7) Universality without document structure dependency;

[0047] Unlike other methods that heavily rely on the hierarchical structure or specific grammatical rules of a document, the method of this invention defines semantic blocks of a document through topological structure, independent of whether the document has explicit paragraph, chapter, or other structure. Therefore, it can adapt to various document types; that is, from news articles to legal documents, technical documents, and patent specifications, it can generate high-quality summaries. This characteristic makes this invention highly versatile, capable of processing multiple text types across languages ​​and domains.

[0048] (8) The advantage of avoiding structural limitations;

[0049] Traditional hierarchical methods and graph neural network-based methods typically require documents to have a certain structure (such as clear paragraph or chapter divisions). However, many documents (such as news reports, social media content, or certain legal documents) are not structurally clear, making them difficult for other methods to process effectively. This invention, through the flexibility of its topological structure, can adapt to these loosely structured documents, generating more accurate and coherent summaries.

[0050] In summary, this invention effectively solves the problems of redundant information, semantic repetition, and information omission in the prior art through the construction and iterative update mechanism of document topology structure. It has obvious technical advantages and is not limited by document type, structure, or language, thus having wide applicability. Attached Figure Description

[0051] Figure 1This is a flowchart of an iterative text summarization method based on document topology, as described in this invention. Detailed Implementation

[0052] Specific Implementation Method 1: Combination Figure 1 This embodiment describes an iterative text summarization system based on document topology. The system includes a text segmentation module, a generation model module, a context fusion module, and a final summary output module. The input document is first processed by the text segmentation module, and the segmentation results are passed to the generation model module for initial summary generation. The context fusion module, located between the text segmentation module and the generation model module, is responsible for fusing the previous summary with the current semantic block to generate a context-enhanced representation, which is used by the generation model module for summary updates in each round. The final summary output module performs final corrections on the output summary. The entire system is deployed on a high-performance GPU server, enabling large-scale parallel processing and real-time summary generation.

[0053] The text segmentation module achieves automatic document segmentation through preprocessing and constructs a topological semantic network of the document based on semantic similarity;

[0054] The similarity of sentences or paragraphs in the input text is calculated using semantic similarity algorithms (such as similarity methods based on cosine similarity or BERT sentence vector calculation). The document is then automatically divided into several semantic blocks with independent semantic units based on a preset similarity threshold or dynamic segmentation algorithm.

[0055] The topological semantic network analyzes the relationships between semantic blocks to form a semantic topology graph of the entire document. This graph describes the logical connections, topic shifts, and information dependencies between semantic blocks, providing global information guidance for subsequent summarization iterations.

[0056] The generation model module uses an existing advanced generation model to perform preliminary summary generation and subsequent summary update tasks;

[0057] In this embodiment, a generative model (e.g., PEGASUS, T5-PEGASUS, or MBART) module is used to generate an initial summary for the first semantic block. This summary effectively captures the core semantics of the block.

[0058] The previous round of summary text and the current semantic block text are concatenated in a preset order to form a joint input sequence, and the joint input sequence is encoded to obtain a context-enhanced representation.

[0059] For the i-th semantic block (i>1), the previous round summary generated by the generative model will be used. With the current semantic block The context fusion module performs joint encoding on the two to obtain a context-enhanced representation; then, the context-enhanced representation is input into the generative model module, which generates an updated summary. This fusion process ensures that the newly generated summary contains both the core information of the current semantic block and retains the global key information from the previous summary, avoiding duplication and omission.

[0060] The final summary output module performs syntax optimization, structural correction, and factual consistency verification on the iteratively generated summary.

[0061] After iterative processing of all semantic blocks, the system outputs a final summary that integrates all local semantic information, has high information density, and is logically coherent. .

[0062] The summary generation system described in this embodiment divides documents into multiple semantic blocks based on semantic similarity and constructs a topological semantic network based on the logical relationships and topic dependencies between semantic blocks. This topological semantic network enables fine-grained semantic segmentation of documents, effectively capturing global semantic information. An iterative update mechanism is proposed: in each round of summary generation, the context fusion module first jointly encodes the previous round's summary and fuses the current semantic block with contextual information to obtain a context-enhanced representation. Then, the generation model module generates an updated summary based on this context-enhanced representation, thereby progressively refining the summary content during the generation process and ensuring global consistency and semantic accuracy.

[0063] The summary generation system described in this embodiment is not limited by document type, language, or structure, and has strong versatility. It generates summaries by fully utilizing the topological structure of documents, that is, the inherent semantic relationships between semantic units (such as sentences or paragraphs) in the document.

[0064] Specific Implementation Method Two: Combination Figure 1 This embodiment describes a method for generating text summarization based on document topology, as described in Specific Embodiment 1. This method first semantically segments the input document, constructs a topological semantic network based on the semantic similarity between sentences or paragraphs, then generates local summaries within each semantic block using a generative model, and integrates contextual information between semantic blocks through an iterative update mechanism to gradually generate a final summary reflecting the global semantics. This method not only optimizes information compression and redundancy filtering but also ensures the accuracy of semantic expression and the coherence of the overall summary. The method is implemented through the following steps:

[0065] Step 1: Document Input and Preprocessing: Receive the input document and perform text cleaning and sentence segmentation using natural language preprocessing tools;

[0066] Step 2: Semantic Blocking and Topology Construction: Using semantic similarity measurement, the document is divided into several semantic blocks of independent semantic units, and the corresponding topological semantic network is constructed.

[0067] Step 3: Initial Summary Generation: For the first semantic block, the generation model module is invoked to generate an initial summary. ;

[0068] Step 4: Iterative Fusion Update: During the iterative fusion update process, the context fusion module receives the summary generated in the previous round and the content of the current semantic block, performs joint representation learning on the two, and filters the effective information in the current semantic block based on the semantic importance of the current semantic block, its relevance to the previous round's summary, and the presence of duplicate information, generating the updated summary for the current round. The context fusion module is positioned between the text segmentation module and the generation model module, serving as a pre-fusion unit for the generation model module. It fuses the information from the previous round's summary with the information in the current semantic block into a context-enhanced representation; the generation model module outputs the updated summary based on this context-enhanced representation. This approach reduces the introduction of duplicate information during the block-by-block processing of document content and improves the semantic coherence and information completeness of the final summary.

[0069] Step 5: Final Summary Output Module: The final summary is processed using syntax rules and factual consistency checks. Perform post-processing to output a high-quality summary.

[0070] In the above process, each module is executed in strict accordance with the predetermined sequence, ensuring efficient information fusion and semantic consistency during the summary generation process.

[0071] The method described in this embodiment systematically solves the technical problems existing in the prior art, such as excessive redundant information, inaccurate semantic expression, insufficient global semantic capture, and poor adaptability to low resources, by constructing a document semantic topology structure, performing block processing, iterative summarization generation based on a pre-trained model, and using a context fusion mechanism. It has significant technical advantages and practical application value.

[0072] The method described in this embodiment has the following advantages:

[0073] (1) Document topology-driven chunking;

[0074] This invention utilizes the inherent semantic relationships within documents to construct a multidimensional topological network, dividing documents of any type into several semantically coherent blocks based on semantic continuity and content density. This block-segmentation process not only effectively reduces redundant information in the original text but also creates relatively concise semantic expression units for subsequent summary generation, thereby significantly improving coding efficiency and model stability.

[0075] (2) Iterative semantic integration and dynamic update mechanism;

[0076] During the iterative summarization process, the model first performs preliminary semantic compression on the first block to generate a preliminary summary. This preliminary summary is then merged with the content of the next block, and the obtained semantic representation is continuously corrected and improved through iterative updates. This dynamic integration mechanism not only fully captures the global and local semantic information of the document but also effectively corrects semantic deviations and incorrect choices in the earlier generation process during continuous iteration, thereby constructing a final summary with higher consistency and more complete information.

[0077] (3) Universality in adapting to various text structures and multilingual environments;

[0078] The iterative summarization method proposed in this invention overcomes the limitations of traditional abstract summarization methods when dealing with texts that are structurally ambiguous or verbose by dynamically learning and refining semantic units of the text block by block. Whether it is academic papers with a clear structure, legal documents, news reports, or operational guidelines, this invention can achieve stable and efficient summary generation and demonstrates excellent universality in multilingual environments (including Chinese, English, French, etc.).

[0079] (4) Information filtering and redundancy suppression during the iterative update process;

[0080] In the iterative summary generation process, this invention performs joint processing of the previous summary and the current semantic block, and combines semantic relevance and duplicate information detection to filter and integrate the effective information in the current semantic block, thereby achieving a good balance between information coverage, conciseness and coherence in the final generated summary.

[0081] (5) Excellent summarizing ability in low-resource scenarios;

[0082] Through block processing and iterative optimization, this invention can still achieve good summarization results under low-resource sample conditions. This innovation not only improves the model's adaptability to new vocabulary generation, but also significantly reduces the training dependence on large-scale data, thereby expanding the feasibility and economy of the method in practical application environments.

[0083] Specific Implementation Method 3: This implementation method is an example of the iterative text summarization method based on document topology structure described in Specific Implementation Method 2.

[0084] (1) Introduction to the dataset;

[0085] This embodiment selects nine representative public datasets to cover various application scenarios and language environments, specifically including:

[0086] The WikiHow dataset: This dataset primarily originates from online knowledge bases and is authored by multiple authors. The texts are quite lengthy, with each article containing multiple steps and a concise summary. This dataset demonstrates the ability to capture the guiding semantics of long text operations.

[0087] The IlPost dataset consists of news summary data from Italian news websites. The text content covers multiple news categories, reflecting the requirements for extracting key information and summarizing key points in news texts.

[0088] The CAIL2020 dataset (judicial abstracts) consists of documents from the Chinese legal field. The texts are lengthy and complex, emphasizing a compressed summary of legal facts, arguments, and the basis for judgments.

[0089] The OrangeSum (abstract / title) dataset includes two tasks: abstract and title. The articles are mainly taken from the French website Orange Actu, and the abstract texts are written by professional authors, requiring highly condensed news reports.

[0090] MLSUM(de) dataset: This dataset is collected from online newspapers and its main task is to generate summaries of the first paragraph of news articles. It has a clear article structure and high information density.

[0091] The WikiLingua (es / tu / vi) dataset contains user guides and documentation in multiple languages, including Vietnamese, Spanish, and Turkish. The model's linguistic universality and robustness are tested through cross-language summary generation.

[0092] Each dataset exhibits significant differences in article length, number of sentences, vocabulary size, and compression ratio between articles and summaries, providing ample experimental conditions for the applicability and diversity of the proposed iterative summarization framework across various fields.

[0093] (2) Evaluation indicators;

[0094] To comprehensively evaluate the accuracy and robustness of the generated summaries, this implementation method employs the following two types of evaluation metrics:

[0095] Automated Metrics (Rouge and FactCC): 1) Rouge Metrics: Rouge-1, Rouge-2, and Rouge-L are selected to measure the degree of matching between the generated summary and the reference summary in terms of vocabulary, pairs of words, and longest common subsequence. Rouge-1 primarily reflects the completeness of word information capture in the generated text; Rouge-2 reflects the matching of pairs of words; and Rouge-L assesses the overall structure and coherence of the summary. 2) FactCC: To verify the performance of the generated summary in terms of factual consistency, FactCC is introduced as an auxiliary tool. FactCC uses a pre-trained BERT model fine-tuned by binary classification to determine the degree of factual matching between the generated text and the source document. Its weighted accuracy and F1 score can intuitively reflect the problem of false information that occurs during semantic extraction.

[0096] Human evaluation: In conjunction with expert review, the abstract texts are comprehensively scored across dimensions such as readability, information completeness, semantic accuracy, and originality. Human evaluation not only verifies the data trends reflected by automated indicators but also compensates for potential shortcomings of purely quantitative assessments in terms of natural language and logical coherence.

[0097] (3) Main experimental results;

[0098] The experimental results of the method of this invention on all datasets are shown in Table 1, with Table 1 being the main experimental results. These results surpass the existing best performance, demonstrating its superiority.

[0099] Table 1

[0100]

[0101] The existing publicly available literature (prior art) cited in the table above is as follows:

[0102] Existing Technique 1: Savelieva, A., Au-Yeung, B., Ramani, V. (2020). Generative summarization of spoken and written instructions with BERT. In KDD 2020 workshop on conversational systems towards mainstream adoption. https: / / arxiv.org / abs / 2008.09676. https: / / arxiv.org / abs / 2008.09676 .).

[0103] Prior art 2: Dou, Z, Liu, P, Hayashi, H, Jiang, Z, Neubig, G. (2020) GSum: A general framework for guided neural generative summarization. Presented at the 2021 North American Association for Computational Linguistics Annual Meeting. https: / / arxiv.org / abs / 2010.08014 .(Dou, Z., Liu, P., Hayashi, H., Jiang,Z., & Neubig, G. (2020). GSum: A general framework for guided neuralabstractive summarization. In 2021 Annual conference of the north american chapter of the association for computational linguistics. https: / / arxiv.org / abs / 2010.08014 .).

[0104] Existing technique 3: Lewis, M, Liu, Y, Goyal, N, Ghazvininejad, M, Mohamed, A, Levy, O, Stoyanov, V, Zettlemoyer, L. (2020) BART: A Denoising Sequence-to-Sequence Pre-training Method for Natural Language Generation, Translation, and Understanding. Presented at the 58th Annual Meeting of the Association for Computational Linguistics. http: / / dx.doi.org / 10.48550 / arXiv.1910.13461.(Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M.,Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020). BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In The 58th annual meeting of the association for computational linguistics. http: / / dx.doi.org / 10.48550 / arXiv.1910.13461 .).

[0105] Prior Art 4: Zhang, J, Zhao, Y, Saleh, M, Liu, PJ (2020) PEGASUS: A pre-training method for generative summarization based on extracting missing sentences. Published at the International Conference on Machine Learning, pp. 11328–11339. http: / / dx.doi.org / 10.48550 / arXiv.1912.08777 . (Zhang, J., Zhao, Y., Saleh, M., & Liu, PJ (2020). PEGASUS: Pre-training with extracted gap-sentences for abstract summarization. In International conference on machine learning(pp.11328–11339). http: / / dx.doi.org / 10.48550 / arXiv.1912.08777 .).

[0106] Prior art 5: Jingun, K, Hidetaka, K, Manabu, O. (2023) An abstract document summarization method incorporating summary length prediction. In the Proceedings of the Association for Computational Linguistics, pp. 618–624. https: / / aclanthology.org / 2023.findings-eacl.45. (Jingun, K., Hidetaka, K., & Manabu, O. (2023). Abstractive document summarization with summary-length prediction. In Findings of the association for computational linguistics (pp. 618–624). https: / / aclanthology.org / 2023.findings-eacl.45 .).

[0107] Prior Art 6: Sarti, G, Nissim, M. (2024) IT5: A Text-to-Text Pre-training Approach for Italian Language Understanding and Generation. In the Proceedings of the Joint International Conference on Computational Linguistics, Language Resources and Evaluation 2024, pp. 9422–9433. https: / / aclanthology.org / 2024.lrec-main.823 .(Sarti, G., &Nissim, M. (2024). IT5: Text-to-text pretraining for Italian language understanding and generation. In Proceedings of the 2024 joint international conference on computational linguistics, language resources and evaluation(pp. 9422–9433). https: / / aclanthology.org / 2024.lrec-main.823 .).

[0108] Existing technique 7: Xue, L, Constant, N, Roberts, A, Kale, M, Al-Rfou, R, Siddhant, A, Barua, A, Raffel, C. (2021) mT5: A pre-trained text-to-text conversion model for large-scale multilingual languages. In Proceedings of the 2021 Association for Computational Linguistics in North America: Human Language Technologies, pp. 483–498. https: / / arxiv.org / abs / 2010.11934. (Xue, L., Constant, N., Roberts, A., Kale, M., Al-Rfou, R., Siddhant, A., Barua, A., & Raffel, C. (2021). mT5: Amassively multilingual pre-trained text-to-text transformer. In Proceedings of the 2021 conference of the north American chapter of the association for computational linguistics: Human language technologies (pp.483–498). https: / / arxiv.org / abs / 2010.11934 .).

[0109] Prior Art 8: Yinhan, L, Jiatao, G, Naman, G, Xian, L, Sergey, E, Marjan, G, Mike, L, Luke, Z. (2020) A Multilingual Denoising Pre-training Method for Neural Machine Translation. Proceedings of the Association for Computational Linguistics, Vol. 8, pp. 726–742. http: / / dx.doi.org / 10.1162 / tacl_a_00343 . (Yinhan, L., Jiatao, G., Naman, G., Xian, L., Sergey, E., Marjan, G., Mike, L., & Luke, Z. (2020). Multilingual denoising pre-training for neural machine translation. Transactions of the Association for Computational Linguistics, 8, 726–742. http: / / dx.doi.org / 10.1162 / tacl_a_00343 .).

[0110] Existing technology 9: Moreno, L. Q, Cagliero, L. (2023) BART-IT: An efficient sequence-to-sequence model for summarizing Italian text. Future Internet, Vol. 15, No. 1, p. 15. http: / / dx.doi.org / 10.3390 / fi15010015. (Moreno, LQ, & Cagliero, L. (2023). BART-IT: An efficient sequence-to-sequence model for Italian text summarization. FutureInternet, 15(1), 15. http: / / dx.doi.org / 10.3390 / fi15010015.).

[0111] Prior Art 10:

[0112] Li, D, Nan, Y, Wenhui, W, Furu, W, Xiaodong, L, Yu, W, Jianfeng, G, Ming, Z, Hsiao-Wuen, H. (2019) A method for pre-training a unified language model for natural language understanding and generation. Advances in Neural Information Processing.

[0113] https: / / proceedings.neurips.cc / paper_files / paper / 2019 / file / c20bb2d9a50d5ac1f713f8b34d9aac5a-Paper.pdf. (Li, D., Nan, Y., Wenhui, W., Furu, W., Xiaodong, L., Yu, W., Jianfeng, G., Ming, Z., & Hsiao-Wuen, H. (2019). Unified language model pre-training for natural language understanding and generation. Advances in Neural Information Processing Systems, https: / / proceedings.neurips.cc / paper_files / paper / 2019 / file / c20bb2d9a50d5ac1f713f8b34d9aac5a-Paper.pdf.).

[0114] Prior art 11: Devlin, J, Chang, M.-W, Lee, K, Toutanova, K. (2019) BERT: A deep bidirectional transformer pre-training method for language understanding. In Proceedings of the 2019 Association for Computational Linguistics: Human Language Technologies Conference, Vol. 1, pp. 4171–4186. https: / / aclanthology.org / N19-1423 . (Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. InProceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1(pp. 4171–4186). https: / / aclanthology.org / N19-1423 .).

[0115] Prior Art 12: Jingpei, D, Weixuan, H, Yuming, W. (2023) Integrating Semantic and Structural Information to Enhance Legal Judgment Summary Generation. Artificial Intelligence and Law, pp. 1–22. http: / / dx.doi.org / 10.1007 / s10506-023-09381-8 .(Jingpei, D., Weixuan, H., & Yuming, W. (2023). Enhancing legal judgment summarization with integrated semantic and structural information. Artificial Intelligence and Law, 1–22. http: / / dx.doi.org / 10.1007 / s10506-023-09381-8 .).

[0116] Prior Art 13: Su, J. (2021) T5 Pegasus - ZhuiyiAI: Technical Report. https: / / github.com / ZhuiyiTechnology / t5-pegasus . (Su, J. (2021). T5 Pegasus - ZhuiyiAI:Technical report, URL https: / / github.com / ZhuiyiTechnology / t5-pegasus .).

[0117] Prior Art 14: Yu, S, Yumei, S, Yongbin, Q, Ruizhang, H, Yanping, C. (2024) A method for generating judgment document summaries based on trial logic steps. Computer Engineering and Applications, Vol. 60, pp. 113–121. http: / / kns.cnki.net / kcms / detail / 11.2127.TP.20230224.0934.004.html .(Yu, S., Yumei, S., Yongbin, Q., Ruizhang, H., & Yanping, C. (2024). Method forgenerating summary of judgment documents based on trial logic steps. ComputerEngineering and Applications, 60,113–121, http: / / kns.cnki.net / kcms / detail / 11. 2127.TP.20230224.0934.004.html .).

[0118] Prior art 15:

[0119] Sascha, R., Shashi, N., Aliaksei, S. (2020) Sequence generation task using pre-trained checkpoints. Proceedings of the Association for Computational Linguistics, Vol. 8, pp. 264–280. http: / / dx.doi.org / 10.1162 / tacl_ a_00313 . (Sascha, R., Shashi, N., & Aliaksei, S. (2020). Leveraging pre-trained checkpoints for sequence generation tasks. Transactions of the Association for Computational Linguistics, 8, 264–280. http: / / dx.doi.org / 10.1162 / tacl_a_00313 .).

[0120] Prior art 16: Eddine, M. K, Tixier, A, Vazirgiannis, M. (2021a) BARThez: A high-performance French sequence-to-sequence pre-trained model. In the Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 9369–9390. http: / / dx.doi.org / 10.48550 / arXiv.2010.12321. (Eddine, MK, Tixier, A., & Vazirgiannis, M. (2021a). BARThez: a skilled pretrained frenchsequence-to-sequence model. In Proceedings of the 2021 conference onempirical methods in natural language processing (pp. 9369–9390). http: / / dx.doi.org / 10.48550 / arXiv.2010.12321 .).

[0121] Prior art 17:

[0122] Gehrmann, S et al. (2021) GEM Benchmark: Natural Language Generation, Its Evaluation and Measurement Methods. In Proceedings of the 1st Workshop on Natural Language Generation, Evaluation and Measurement Methods, pp. 96–120. http: / / dx.doi.org / 10.48550 / arXiv.2102.01672 (Gehrmann, S., Adewumi, T., Aggarwal, K., Ammanamanchi, PS, Anuoluwapo, A., Bosselut, A., Chandu, KR, Clinciu, M., Das, D., Dhole, KD, Du, W., Durmus, E., Dušek, O., Emezue, C., Gangal, V., Garbacea, C., Hashimoto, T., Hou, Y., Jernite, Y., .... Zhou, J. (2021). The GEM benchmark: Natural language generation, its evaluation and metrics. In Proceedings of the 1st workshop on natural language generation, evaluation, and metrics (pp.96–120). http: / / dx.doi.org / 10.48550 / onXiv.2102.01672 .).

[0123] In this embodiment, experimental results using the ROUGE metric demonstrate that the method of this invention effectively utilizes document structure, particularly on the WikiHow and WikiLingua datasets, significantly improving summary quality by identifying semantic variations. For example, on the WikiHow dataset, it improves ROUGE-1, ROUGE-2, and ROUGE-L by 4.13, 3.64, and 5.00, respectively. When processing structured text, such as the MLSUM(de) dataset, the method improves summary quality through segmentation and semantic processing. It improves ROUGE-1, ROUGE-2, and ROUGE-L by 0.7, 1.93, and 1.53, respectively. The method of this invention performs excellently in long text summarization, especially on the CAIL2020(sfzy) dataset, improving ROUGE-1, ROUGE-2, and ROUGE-L by 10.60, 16.49, and 8.55, respectively. Our method also performs well when processing complex text, such as the IlPost and OrangeSum datasets, improving ROUGE scores accordingly.

[0124] On the FactCC metric, the method of this invention also outperformed all previous models, and the generated summary was consistent with the facts of the source document, demonstrating its high quality.

[0125] In summary, the method of this invention demonstrates strong generalization ability and can achieve excellent results on a variety of datasets.

[0126] (4) Ablation test results;

[0127] To further explore the impact of this invention on abstract quality, an ablation experiment was designed. Specifically, after segmenting the article, an abstract was generated for each segment, and then these results were spliced ​​together to form the final abstract. The experimental results are shown in Table 2, which presents the ablation experiment results.

[0128] Table 2

[0129]

[0130] The experimental results clearly show a significant decrease in the performance of the ablation experiment. The reasons for this phenomenon are as follows:

[0131] Independence in summary generation: In the ablation experiments, summaries were generated separately for each paragraph. Each generation introduced new semantic information for the model, but lacked guidance from older semantics, resulting in a significant cost for convergence. This discontinuity led to a significant drop in model performance because the lack of continuous contextual support resulted in incoherent generated summaries.

[0132] Disruption of Information Flow: Ablation experiments, by processing the article in multiple parts, disrupt the information flow, preventing the model from capturing semantic dependencies between different paragraphs. This fragmentation of information makes it impossible for the summary of each paragraph to be effectively combined with the summaries of other paragraphs, affecting the overall quality of the summary.

[0133] Insufficient semantic integration capability: Ablation experiments failed to compress and summarize existing semantics while integrating new semantics. Because each paragraph was processed independently, the lack of a comprehensive summary of the entire article's semantics led to a significant drop in model performance. The model did not fully utilize the connections between paragraphs when synthesizing the final summary, thus reducing the overall effectiveness of the summary.

[0134] The results and analysis of ablation experiments conclude that effective integration of document information is crucial for abstract generation. This invention, by segmenting the article, not only efficiently compresses and summarizes document content but also continuously reduces the interference of document content on the abstract generation process, thus demonstrating its strong stability and adaptability.

[0135] (5) Low-resource experiments;

[0136] In practical applications, it is often difficult to collect large amounts of supervised data for model training. Therefore, low-resource summarization has become an important research area. To this end, the first 10 and 100 samples were selected from the training set for training. Since previous research on low-resource summarization has mostly focused on mainstream languages ​​such as English, with less attention paid to other languages, MBART and MT5 models were chosen for comparative experiments. The results of these experiments are shown in Table 3, which presents the low-resource experiment results.

[0137] Table 3

[0138]

[0139] Experimental results show that IterSum still achieves relatively leading performance even in low-resource environments. This is because IterSum has the ability to effectively integrate new semantics while compressing old semantics, and this method remains effective even with a small number of samples. Therefore, IterSum still demonstrates superior summarization capabilities in low-resource environments.

[0140] In summary, this invention demonstrates significant advantages in summarizing quality, module efficiency, and resource utilization through main experiments, ablation experiments, and low-resource experiments. This method not only maintains high-precision summarizing performance in multi-domain and multilingual environments but also effectively reduces computational complexity and energy consumption through a block-based iterative strategy, exhibiting strong robustness and scalability. It provides an innovative technical solution with significant application value for the efficient structured understanding and accurate summarizing of complex texts.

[0141] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0142] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. An iterative text summarization system based on document topology, characterized by: The generation system includes a text segmentation module, a generation model module, a context fusion module, and a final summary output module; After the input document is processed by the text segmentation module, the segmentation results are sent to the generation model module for initial summary generation. The context fusion module is used to iteratively update the initial summary output by the generation model module in each round; that is: receiving the summary generated in the previous round and the content of the current semantic block, performing joint representation learning on the two, and filtering the effective information in the current semantic block according to the semantic importance in the current semantic block, the relevance to the previous round summary and the repetition of information, and generating the updated summary for the current round. The final summary output module performs a final correction on the summary output by the context fusion module.

2. The iterative text summarization system based on document topology according to claim 1, characterized in that: The text segmentation module achieves automatic document segmentation through preprocessing and constructs a topological semantic network of the document based on semantic similarity. It uses a semantic similarity algorithm to calculate the similarity of sentences or paragraphs in the input text and automatically divides the document according to a preset similarity threshold or a dynamic segmentation algorithm to obtain several semantic blocks with independent semantic units.

3. The iterative text summarization system based on document topology according to claim 2, characterized in that: The generative model module generates an initial summary for the first semantic block, which effectively captures the core semantics of the semantic block.

4. The iterative text summarization system based on document topology according to claim 3, characterized in that: The context fusion module is set between the text segmentation module and the generation model module, and serves as a pre-fusion unit of the generation model module. It is used to fuse the previous round of summary information with the current semantic block information into a context-enhanced representation. The generative model module outputs an updated summary based on the context-enhanced representation.

5. The iterative text summarization system based on document topology according to claim 4, characterized in that: The final summary output module performs syntactic optimization, structural correction, and factual consistency verification on the iteratively generated summary; and generates a final summary that integrates all local semantic information, has high information density, and logical coherence.

6. An iterative text summarization method based on document topology, characterized by: This method is implemented by the following steps: Step 1: The text segmentation module performs semantic segmentation and topology construction on the input document; By using semantic similarity calculation, the input document is divided into several semantic blocks of independent semantic units, and the corresponding topological semantic network is constructed. Step 2: Initial Summary Generation For the first semantic block obtained in step one, an initial summary is generated using the generative model module. ; Step 3, Iterative Fusion Update: For each subsequent semantic block, the summary generated in the previous round is jointly encoded with the current semantic block and then input into the context fusion module. The context fusion module extracts the key information in the current semantic block and fuses it with the global semantic information in the previous round summary to obtain the context-enhanced representation. Then, the generation model module generates an updated summary based on the context-enhanced representation until all semantic blocks are processed. Step 4: Output the final summary through the final summary output module. .

7. The iterative text summarization method based on document topology according to claim 6, characterized in that: In step one, the text segmentation module is also used to perform text cleaning and sentence segmentation on the input document using natural language preprocessing tools.

Images