Cross-language long document summarization method based on dynamic latent key information constraints

Abstract

The present application relates to a cross-language long document summarization method based on dynamic latent key information constraint, and belongs to the technical field of natural language processing. The method comprises: cross-language long document summarization dataset construction; cross-language long document summarization model construction based on dynamic latent key information constraint; cross-language long document summarization model training based on dynamic latent key information constraint; and cross-language summarization generation for long document content. According to the four-part method process, a cross-language long document summarization device based on dynamic latent key information constraint is modularly manufactured; the present application can effectively mine deep semantic association and structural features in long documents, and solve the semantic drift and structural defocus problems existing in the traditional method in the cross-language scene. In long document processing, the present application exhibits a significant performance advantage, and the performance of the present application in the ROUGE-L index is improved compared with the baseline model, thereby providing technical support for multilingual information integration, international knowledge sharing and the like.

Description

Technical Field

[0001] This invention relates to a cross-language long document summarization method based on dynamic potential key information constraints, and belongs to the field of natural language processing technology. Background Technology

[0002] The core objective of cross-lingual long document summarization is to compress long documents in the source language into summaries in the target language, enabling accurate and efficient extraction of key information. With deepening globalization and the explosive growth of digital information, the number of multilingual long documents continues to increase, covering an increasingly diverse range of languages. The expansion in text size, such as news reports, academic papers, and business reports, makes efficient processing of long documents a significant challenge, especially in applications like multinational corporate collaborations and the construction of multilingual knowledge bases. However, traditional human translation and summarization methods have significant limitations; they are not only time-consuming and labor-intensive but also struggle to balance real-time performance, accuracy, and cost-effectiveness, failing to meet current information processing needs. Therefore, how to automatically generate high-quality long document summaries in cross-lingual environments has become an important research direction in the field of natural language processing, showing broad prospects in practical applications.

[0003] Current research on cross-lingual summarization mainly focuses on short documents such as news articles. Multilingual pre-trained language models have achieved good results in cross-lingual summarization tasks due to their ability to effectively capture cross-lingual features and semantic information. However, cross-lingual long document summarization not only involves the problem of information compression of long documents but also requires ensuring cross-lingual semantic consistency, making the direct generation of high-quality cross-lingual long document summaries extremely challenging. Currently, there are two main approaches to dealing with extremely long input text. One is to improve the attention mechanism so that the model can handle longer text content. However, in the context of cross-lingual long documents, due to the large amount of redundant and irrelevant information in the text, and the fact that key information is scattered throughout the document, coupled with the forgetting problem caused by long-distance dependencies, the model struggles to effectively extract complete key information, potentially leading to semantic bias and content loss. The second approach is to use hierarchical modeling, dividing the long document into smaller chunks that the model can directly process, and finally obtaining the representation of the entire document through layer-by-layer modeling. Although this method can alleviate computational pressure and improve summarization quality to some extent, it still faces many challenges. First, the reasonable division of text segments is crucial. Segmentation strategies must maintain contextual coherence while covering key information; otherwise, information fragmentation or missing context may occur, affecting the completeness of the summary. Secondly, hierarchical modeling relies on information aggregation at multiple stages; how to efficiently integrate key information across different levels while minimizing noise interference remains an unresolved issue. Furthermore, in cross-linguistic scenarios, hierarchical modeling methods must ensure semantic alignment without deviation during information transmission across different levels to reduce the degradation of summary quality caused by translation inconsistencies or information loss.

[0004] To address the aforementioned problems, this invention proposes a cross-language long document summarization method based on dynamic potential key information constraints. Summary of the Invention

[0005] The technical problem this invention aims to solve is as follows: This invention provides a cross-lingual long document summarization method based on dynamic potential key information constraints. This addresses the issue that mainstream cross-lingual summarization methods are limited by the input length of pre-trained language models, typically employing segmentation or text truncation strategies to handle long document content, leading to incomplete modeling of long document information. Furthermore, the compromised structural integrity of long texts and the breakage of cross-paragraph logical chains further increase the difficulty of aligning cross-lingual semantic spaces, resulting in incomplete information in the generated summaries. The method of this invention has significant advantages in terms of the coherence, information coverage, and fluency of the generated summaries.

[0006] The technical solution of this invention is: a cross-language long document summarization method based on dynamic potential key information constraints, the method comprising:

[0007] Step 1: Construction of a cross-language long document summarization dataset: A method for constructing cross-language long document summarization based on a large language model with a prompting mechanism is proposed, and a round-trip translation strategy is introduced to construct a cross-language long document summarization dataset with high semantic consistency.

[0008] Step 2: Construction of a cross-language long document summarization model based on dynamic latent key information constraints: By transforming the cross-language long document summarization task into a method for extracting dynamic key information and generating a hybrid source language summary and target language summary, the model's ability to fully model the semantic information of long documents is improved; and the source language summary is introduced as auxiliary information to enhance cross-language semantic consistency through joint decoding.

[0009] Step 3: Training of cross-language long document summarization model based on dynamic potential key information constraints: Introducing a consistency loss function and achieving end-to-end joint optimization through gradient collaboration mechanism to generate more coherent and accurate cross-language long document summaries;

[0010] Step 4: Generate cross-language summaries of long document content: Build a complete cross-language long document summarization prototype system; using the Sanic and VUE technology frameworks, implement a complete functional flow where users submit long documents through the front-end interface, the back-end calls the model to generate summaries, and the processing results are displayed to users in an intuitive way.

[0011] Further, Step 1 includes:

[0012] Step 1.1: Propose a large language model translation method based on prompting mechanism to convert English reference summaries in the SumSurvey long document summaries dataset into Chinese cross-language summaries;

[0013] Step 1.2: Use a machine translation system to back-translate the generated Chinese cross-language summary to obtain the corresponding English back-translated summary;

[0014] Step 1.3: Assess the similarity between the English back-translated abstract and the original English reference abstract, and calculate the ROUGE-1 and ROUGE-2 scoring indicators; compare the scoring indicators with the threshold, retain the abstract pairs that meet the threshold, and discard the rest.

[0015] Furthermore, Step 2 includes:

[0016] Step 2.1: Construct a dynamic key information extractor module: This module models sentences in the input long document separately based on a pre-trained semantic model. By scoring the sentences, it selects the combination of sentences with the highest scores, thereby realizing the automatic identification and extraction of key information sentences in the document.

[0017] Step 2.2: Construct a cross-language summarization generator module: This module takes the source language summary as a prior condition and adopts a shared encoder architecture to model both the source language summary and the target language summary simultaneously.

[0018] Further, Step 2.1 includes:

[0019] Ste2.1.1 In the cross-language long document summarization task, consecutive text segments are grouped and the encoding vector of each segment is calculated separately;

[0020] Step 2.1.2: Map the encoded vector to the scalar score using a Multilayer Perceptron (MLP). ,in, These are the parameters of the extractor. For query vector, Indicates the first A text fragment, The parameter is The scoring function of the dynamic key information extractor module;

[0021] Step 2.1.3, Based on scalar scores Sort the segments of the entire document and select the highest-scoring ones. A fragment As input for summary generation:

[0022] ;

[0023] in, A collection of all fragments in the document; The preset number of sentence segments, To select the highest score A fragment.

[0024] Further, Step 2.2 includes:

[0025] Step 2.2.1: Generate target summary Replace with source language summary and target language summary The sequence is connected; the specific formula is as follows:

[0026] ;

[0027] in, Source language summary generated for the model , Target language summary generated for the model , and These represent the start and end markers of the output summary, respectively. It is used for separation and Special markings;

[0028] Step 2.2.2: Under the new generation objective, the cross-language summarization generator module will select the highest-scoring module. As input, the decoder first learns to generate Then and Generate under the conditions The specific formula is as follows:

[0029] ;

[0030] in, The model parameters are: Conditional probability distribution of the cross-language summary generator module; Indicates that in a given query and Under these conditions, the model generates the entire target language summary sequence. The total probability, For the first The target language summary sequence generated in the first step; Indicates at time step The previously generated target language summary sequence; This represents the maximum time step.

[0031] Furthermore, Step 3 includes:

[0032] Step 3.1: Select the sentence group with the highest ROUGE score obtained through greedy search. As a supervisory signal, the dynamic key information extractor module extracts sentence groups and... Cross-entropy loss between As part of the loss of the dynamic key information extractor module, the specific calculation formula is as follows:

[0033] ;

[0034] in, The parameter is The scoring function of the dynamic key information extractor module; X is the set of all fragments in the document; Indicates a monitoring signal The number of segments;

[0035] Step 3.2: Calculate the generation loss by combining the generated monolingual and cross-lingual summaries with the referenced monolingual and cross-lingual summaries. The specific calculation method is as follows:

[0036] ;

[0037] in, The parameter is Conditional probability distribution of the cross-language summary generator module; For the first The source language summary sequence generated step by step, For the first The target language summary sequence generated in the first step; , Indicates at time step The previously generated source language and target language summary sequences; Maximum time step;

[0038] Step 3.3: Employ a consistency alignment strategy based on dynamic weights to improve model consistency by constraining the latent space representations of different modules; calculate the probability distribution of the dynamic key information extractor module. Dynamic weights in the cross-language summarization generator module KL divergence between distributions enables cross-module collaborative optimization;

[0039] ;

[0040] in, This indicates that the dynamic key information extractor module extracts fragments. The probability distribution of choice; This represents the distribution of dynamically weighted components in the cross-language summarization generator module. Denotes KL divergence, This represents a measure of the difference between two probability distributions. By measuring the difference between the two probability distributions, the dynamic key information extractor module and the cross-language summary generator module are encouraged to converge in decision-making.

[0041] Furthermore, Step 4 includes:

[0042] Step 4.1: Save the trained cross-language long document summarization model based on dynamic latent key information constraints as a ".pth" file, and load the cross-language long document summarization model based on dynamic latent key information constraints into memory using the Flask framework;

[0043] Step 4.2: Build a complete cross-language long document summarization prototype system to provide efficient core information extraction and summary generation services; the core functions of the system include: long document preprocessing, summary generation based on key information extraction, cross-language summary conversion and result visualization;

[0044] Step 4.3: By calling the cross-language long document summarization model based on dynamic potential key information constraints deployed on the server through the web client, the input long document is summarized, and the generated summary results are displayed on the front-end page in real time, realizing an interactive summary service for users.

[0045] This invention also provides a cross-language long document summarization apparatus based on dynamic latent key information constraints. The apparatus includes a module for executing the aforementioned cross-language long document summarization method based on dynamic latent key information constraints. The beneficial effects of this invention are:

[0046] 1. Currently, cross-linguistic long document summarization datasets specifically designed for the characteristics of long documents are very scarce. To this end, this invention uses existing long document summarization datasets, leverages the prompt generation capabilities of large language models to translate reference summaries, and combines a round-trip translation strategy to ensure translation quality, thereby constructing a cross-linguistic long document summarization dataset to provide data support for subsequent research.

[0047] 2. Existing cross-language summarization methods struggle to maintain accurate cross-semantic alignment during information compression, leading to missing content or semantic bias in the summaries. This invention decomposes the cross-language long document summarization task into a hybrid summarization method combining extractive and generative approaches. By utilizing an extractor to extract key information from the document, document length and information redundancy are reduced, and cross-language summaries are generated based on this extracted information. Simultaneously, by jointly optimizing the extractor and generator, the overall performance of the model is optimized.

[0048] 3. To verify the effectiveness of the method, systematic experiments were conducted on the long document summarization dataset (Cross-SumSurvey). The experimental results show that the proposed model significantly outperforms existing mainstream baseline methods in automatic evaluation metrics such as ROUGE. At the same time, manual evaluation also verified that the method has significant advantages in terms of the coherence, information coverage and fluency of the generated summaries, indicating that the method has good generalizability and practical value in practical applications. Attached Figure Description

[0049] Figure 1 This is a structural diagram of the model of the present invention;

[0050] Figure 2 This is a flowchart illustrating the overall process of the present invention. Detailed Implementation

[0051] Example 1: As Figures 1-2 As shown, a cross-language long document summarization method based on dynamic latent key information constraints includes:

[0052] Step 1: Construction of a cross-language long document summarization dataset: Currently, most cross-language summarization datasets are constructed around short documents such as news articles, and there is a lack of long document summarization datasets for cross-language documents. To address the lack of publicly available cross-language long document summarization datasets, we propose a method for constructing cross-language long document summarization based on a large language model with a prompting mechanism, and introduce a round-trip translation strategy to achieve translation quality control, thereby constructing a cross-language long document summarization dataset with high semantic consistency.

[0053] Step 2: Construction of a Cross-Language Long Document Summarizing Model Based on Dynamic Latent Key Information Constraints: Mainstream cross-language summarizing methods are limited by the input length of pre-trained language models, typically employing segmentation or text truncation strategies to handle long document content, resulting in incomplete modeling of long document information. Furthermore, the structural integrity of long texts is compromised, and the logical chains across paragraphs are broken, further increasing the difficulty of aligning cross-language semantic spaces. Addressing the issues of incomplete modeling of long document information and broken logical chains across paragraphs in current cross-language summarizing methods, this invention proposes a novel cross-language long document summarizing model based on dynamic latent key information constraints. By transforming the cross-language long document summarizing task into a method for extracting dynamic key information and generating a hybrid source language summary and target language summary, the model's ability to comprehensively model the semantic information of long documents is improved. Experimental results show that this invention improves performance on the ROUGE-L metric compared to the baseline model. Additionally, the source language summary is introduced as auxiliary information, and joint decoding enhances cross-language semantic consistency.

[0054] Step 3: Training of cross-language long document summarization model based on dynamic potential key information constraints: In order to optimize the collaborative effect of the extractor and the generator, a consistency loss function is introduced to achieve end-to-end joint optimization through gradient collaboration mechanism, which is used to generate more coherent and accurate cross-language long document summaries, and can effectively optimize the model's performance in both extraction and generation tasks.

[0055] Step 4: Generate cross-language summaries of long documents: Build a complete cross-language long document summarization prototype system to provide efficient core information extraction and summary generation services. The core functions of the system include: long document preprocessing, summary generation based on key information extraction, cross-language summary conversion, and result visualization. With the help of the Sanic and Vue technology frameworks, the system enables users to submit long documents through the front-end interface, the back-end to call the model to generate summaries, and the processing results to be displayed to users in an intuitive way.

[0056] Further, Step 1 includes:

[0057] Step 1.1: Propose a large language model translation method based on prompting mechanism to convert English reference summaries in the SumSurvey long document summaries dataset into Chinese cross-language summaries;

[0058] Step 1.2: Use a machine translation system (such as Yunling Translation http: / / yuntrans.vip or GoogleTranslate) to perform back-translation on the generated Chinese cross-language summary to obtain the corresponding English back-translated summary;

[0059] Step 1.3: Assess the similarity between the English back-translated abstract and the original English reference abstract, and calculate the ROUGE-1 and ROUGE-2 scoring indicators. Compare the scoring indicators with the thresholds, and only retain the abstract pairs with a ROUGE-1 score of not less than 0.45 and a ROUGE-2 score of not less than 0.2. The remaining abstract pairs are eliminated to ensure that the retained translations have high quality in terms of semantic consistency and language expression.

[0060] Furthermore, Step 2 includes:

[0061] Step 2.1: Construct a dynamic key information extractor module: This module models sentences in the input long document separately based on a pre-trained semantic model. By scoring the sentences, it selects the combination of sentences with the highest scores, thereby realizing the automatic identification and extraction of key information sentences in the document and significantly reducing the interference of redundant information on the generation of the summary.

[0062] Step 2.2: Construct a cross-language summarization generator module: This module takes the source language summary as a prior condition and adopts a shared encoder architecture to model both the source language summary and the target language summary simultaneously.

[0063] Further, Step 2.1 includes:

[0064] Ste2.1.1 In cross-language long document summarization tasks, due to GPU memory limitations, it is impossible to directly concatenate all text segments and input them into a large pre-trained language model for encoding. Therefore, we group consecutive text segments and calculate the encoding vector for each segment separately;

[0065] Step 2.1.2: Map the encoded vector to the scalar score using a Multilayer Perceptron (MLP). ,in, These are the parameters of the extractor. For query vector, Indicates the first A text fragment, The parameter is The scoring function of the dynamic key information extractor module;

[0066] Step 2.1.3, Based on scalar scores Sort the segments of the entire document and select the highest-scoring ones. A fragment As input for summary generation:

[0067] ;

[0068] in, A collection of all fragments in the document; The preset number of sentence segments, To select the highest score A fragment.

[0069] Further, Step 2.2 includes:

[0070] Step 2.2.1: Generate target summary Replace with source language summary and target language summary The sequence is connected; the specific formula is as follows:

[0071] ;

[0072] in, Source language summary generated for the model , Target language summary generated for the model , and These represent the start and end markers of the output summary, respectively. It is used for separation and Special markings;

[0073] Step 2.2.2: Under the new generation objective, the cross-language summarization generator module will select the highest-scoring module. As input, the decoder first learns to generate Then and Generate under the conditions The specific formula is as follows:

[0074] ;

[0075] in, The model parameters are: Conditional probability distribution of the cross-language summary generator module; Indicates that in a given query and Under these conditions, the model generates the entire target language summary sequence. The total probability, For the first The target language summary sequence generated in the first step; Indicates at time step The previously generated target language summary sequence; This represents the maximum time step.

[0076] Furthermore, Step 3 includes:

[0077] Ste3.1, select the sentence group with the highest ROUGE score obtained through greedy search. As a supervisory signal, the dynamic key information extractor module extracts sentence groups and... Cross-entropy loss between As part of the loss of the dynamic key information extractor module, the specific calculation formula is as follows:

[0078] ;

[0079] in, The parameter is The scoring function of the dynamic key information extractor module; X is the set of all fragments in the document; Indicates a monitoring signal The loss function minimizes the cross-entropy between the extractor output and the target sentence group, prompting the model to learn to extract segments similar to the reference sentence group. Since the extraction task directly depends on the ROUGE score, this loss function effectively reflects the model's extraction ability in text summarization generation.

[0080] Ste3.2 Since the generator produces a joint output of monolingual and cross-lingual summaries, the loss is calculated by combining the generated monolingual and cross-lingual summaries with the referenced monolingual and cross-lingual summaries as the generation loss. The specific calculation method is as follows:

[0081] ;

[0082] in, The parameter is Conditional probability distribution of the cross-language summary generator module; For the first The source language summary sequence generated step by step, For the first The target language summary sequence generated in the first step; , Indicates at time step The previously generated source language and target language summary sequences; The maximum time step is defined; by maximizing this loss function, the generator learns how to generate text that conforms to the target summary given the input and context. This loss function considers both monolingual and cross-lingual summarization generation, thus enabling the model to handle multilingual capabilities in generation tasks.

[0083] Ste3.3. A consistency alignment strategy based on dynamic weights is adopted to improve model consistency by constraining the latent space representations of different modules; the probability distribution of the dynamic key information extractor module is calculated. Dynamic weights in the cross-language summary generator module KL divergence between distributions enables cross-module collaborative optimization;

[0084] ;

[0085] in, This indicates that the dynamic key information extractor module extracts fragments. The probability distribution of choice; This represents the distribution of dynamically weighted components in the cross-language summarization generator module. Denotes KL divergence, The concept represents the difference between two probability distributions. By measuring the difference between the two probability distributions, the dynamic key information extractor module and the cross-language summary generator module are encouraged to make consistent decisions. The introduction of this consistency loss helps the model achieve collaborative optimization between the two modules, thereby improving the overall quality of summary generation.

[0086] Furthermore, Step 4 includes:

[0087] Step 4.1: Save the trained cross-language long document summarization model based on dynamic latent key information constraints as a ".pth" file, and load the model into memory using the Flask framework to avoid frequent loading caused by multiple requests, thereby improving the model's running efficiency during the inference phase; deploy the model as an API interface based on the Flask framework to support multiple concurrent accesses to the web port and meet the high-concurrency request requirements of practical applications;

[0088] Step 4.2: Build a complete cross-language long document summarization prototype system to provide efficient core information extraction and summary generation services; the core functions of the system include: long document preprocessing, summary generation based on key information extraction, cross-language summary conversion and result visualization;

[0089] Step 4.3: By calling the cross-language long document summarization model based on dynamic potential key information constraints deployed on the server through the web client, the input long document is summarized, and the generated summary results are displayed on the front-end page in real time, realizing an interactive summary service for users.

[0090] The present invention also provides a cross-language long document summarization apparatus based on dynamic latent key information constraints, the apparatus comprising: a module for performing the cross-language long document summarization method based on dynamic latent key information constraints.

[0091] To illustrate the effectiveness of this invention, experimental verification was conducted on the self-constructed cross-domain long document summarization benchmark dataset Cross-SumSurvey, with the data source selected from the internationally authoritative scientific literature aggregation platform SumSurvey. This dataset has an average document length of 2,384 tokens, with one-quarter of the samples exceeding 16,000 tokens and the largest document size reaching 24,576 tokens. It covers three major interdisciplinary fields: computer science (including algorithm theory and system architecture), biomedicine (including clinical trials and molecular biology), and social sciences (including behavioral economics and policy analysis), and includes seven text genres such as empirical research, theoretical deduction, and review articles.

[0092] During the experimental evaluation, this invention uses a dual evaluation dimension of lexical overlap and semantic similarity to systematically verify the generation quality and cross-language alignment capability of the cross-language long document summarization model, specifically employing the following metrics:

[0093] • ROUGE series: Quantify content coverage and structural coherence by generating N-gram recall (ROUGE-N) and longest common subsequence matching (ROUGE-L) between the summary and the reference summary.

[0094] • BLEU: Evaluates local matching accuracy based on N-gram precision and introduces a short sentence penalty factor (BrevityPenalty) to suppress redundant generation.

[0095] • METEOR: Integrates synonym mapping and stemming technology to reconcile precision and recall, and enhance the consistency of human evaluation.

[0096] • CIDEr: Employs a TF-IDF weighting mechanism to enhance the evaluation weight of low-frequency core terms in technical documents.

[0097] • BERTScore: Based on contextual semantic embedding of a pre-trained language model, it calculates the deep semantic alignment between the generated text and the reference text.

[0098] This invention undergoes a systematic comparative analysis with several state-of-the-art models in the fields of extractive and abstract summarization to comprehensively evaluate its performance advantages in long document automatic summarization tasks. For extractive summarization, several representative baseline models were selected, including Longest, TextRank, PacSum, and SummaRu. Among them:

[0099] Longest: Based on heuristics, select the k longest sentences from each document as a summary;

[0100] TextRank: An unsupervised sentence ranking algorithm based on undirected graphs. It constructs a graph structure using cosine similarity between sentences and iteratively calculates the importance of nodes.

[0101] PacSum: Based on TextRank, it introduces sentence position information and transforms an undirected graph into a directed graph by judging the inter-sentence pointing relationship, which significantly improves the accuracy of key sentence extraction;

[0102] SummaRu: A supervised extraction model based on recurrent neural networks (RNNs), using labeled data for end-to-end training. For extractive summarization, two representative baseline models, PGN and LED, were selected.

[0103] Among them, PGN is a seq2seq model based on LSTM, which introduces a copy mechanism and a coverage mechanism to alleviate the problems of out-of-vocabulary words and repeated generation. Although it supports long document input, its long-distance dependency modeling ability is relatively weak due to the limitations of the LSTM architecture.

[0104] LED: A seq2seq model based on a sparse attention mechanism, with weights initialized by BART (a pre-trained Transformer model). Its sliding window attention mechanism can efficiently handle long documents. Additionally, the end-to-end method mLED was selected, which uses a Longformer sparse attention mechanism based on mBART-50 and can handle input texts up to 16K tokens long.

[0105] Experiment 1: Main Experimental Results

[0106] Table 1 shows the main experimental results.

[0107]

[0108] As shown in Table 1, the performance of the three baseline methods exhibits significant hierarchical differences. The pipeline method (Ext-Trans) performs the weakest overall. The best-performing model, SummaRu, significantly lags behind the end-to-end method in BLEU-4 (-1.3) and CIDEr (42.0), verifying the inherent defects of the traditional "extract-translate" paradigm: 1) Sentence-level extraction leads to global semantic fragmentation (SummaRu's ROUGE-L score is only 18.9 vs. Ours' 29.2); 2) Independent translation stages cause error accumulation (Longest's BLEU-4 score is -1.6, indicating that n-gram matching completely fails). The abstract generation method (Abs-Trans) achieves a breakthrough in semantic quality metrics. The LED model, through pre-trained BART weights, improves CIDEr (56.3) and BERTScore (82.6) by 14.3 and 1 respectively compared to SummaRu, proving that the intermediate source language generation stage can effectively model document-level semantics. However, its BLEU-4 (-0.6) is still negative, indicating that the traditional seq2seq architecture is difficult to capture the fine-grained patterns of long documents. The end-to-end approach demonstrates systematic advantages, and the excellent performance of mLED on ROUGE-L (26.7) and CIDEr (64.5) verifies the effectiveness of the sparse attention mechanism for modeling long texts. The method of this invention achieves comprehensive improvement through a three-level collaborative mechanism of hierarchical information filtering, intermediate representation bridging, and joint optimization: 1) In terms of content integrity, ROUGE-L (29.2) improves by 2.5 compared to mLED, reflecting the deep modeling ability of the heterogeneous graph encoder for document structure; 2) In terms of language quality, METEOR (22.2) and BERTScore (84.2) are optimized by 0.5 and 0.6 respectively compared to mLED, proving that the bridging role of monolingual summarization in the generator alleviates cross-linguistic grammatical fragmentation; 3) In terms of generation diversity, BLEU-4 (1.4) achieves a positive value, improving by 0.8 compared to mLED, indicating that consistency loss significantly reduces the n-gram fragmentation problem by constraining semantic alignment between modules. CIDEr improves by 0.7 compared to mLED on a subset of documents with lengths of 16K+, further verifying the adaptability of the hybrid method to ultra-long texts.

[0109] Experiment 2: Ablation Experiment

[0110] To fully verify the effectiveness of the proposed method, ablation experiments were conducted to explore the impact of different modules on overall performance. Here, w / o EX represents removing the extractor module; w / o CL represents removing consistency loss; and w / o SG represents removing source language summary generation, with the generator directly generating cross-language summaries. The experimental results are shown in Table 2. Removing the extractor (w / o EX) resulted in the greatest performance degradation, with the key indicator BLEU-4 decreasing by 1.2 (3.4→2.2), indicating that the extractor's selection of key sentences is fundamental to semantic coherence. Removing source language summary generation (w / o SG) caused ROUGE-L to decrease by 1.8 (35.6→33.8), verifying the necessity of bridging cross-language semantics through monolingual summaries in the intermediate generation stage. Removing consistency loss (w / o CL) led to a BERTScore decrease of 0.7 (86.2→85.5), reflecting the crucial role of explicit semantic alignment constraints on generation quality. Further analysis revealed that the peak performance of the complete model on ROUGE-L (35.6) and CIDEr (67.2) stemmed from the module synergy effect: the extractor filters core information through hierarchical heterogeneous graphs, the generator enhances cross-linguistic alignment using monolingual summarization, and consistency loss ensures that the optimization objectives of the two modules are consistent. The stepwise increase in BLEU-4 (2.2→3.4) indicates that module synergy has a cumulative gain on n-gram diversity, with the low w / o EX values ​​on ROUGE-2 (8.9) and BLEU-3 (4.3) corroborating the loss of fine-grained semantics due to the absence of key sentences. Compared to baseline methods, the complete model achieves a systematic breakthrough in the end-to-end framework, attributed to the deep integration of hierarchical information flow design, intermediate representation bridging, and joint optimization, ultimately achieving a balance between content integrity and language adaptability.

[0111] Table 2 shows the results of the ablation experiment.

[0112]

[0113] Experiment 3: Experiments with different numbers of sentences

[0114] To evaluate the impact of the number of extracted sentences on model performance, this invention tested different numbers of extracted sentences on a dataset. Experimental results show that as the number of extracted sentences (i.e., the K value) increases, the overall model performance exhibits a significant upward trend. This phenomenon aligns with the expectations of this invention, because when the model can extract more sentences, it obtains more comprehensive and richer information, providing stronger support for the subsequent generator and thus generating higher-quality summaries in terms of content coverage and expressive accuracy. In other words, a higher K value gives the model more opportunities to capture key information from the original document, thereby improving the overall performance of the generated summary. Due to the limitations of GPU memory resources, this invention could not increase the number of extracted sentences indefinitely during the experiments. Therefore, this invention set the maximum K value to 25.

[0115] Table 3. Experimental results on the Cross-SumSurvey dataset with different numbers of extracted sentences.

[0116]

[0117] Experiment 4: Artificial Evaluation Experiment

[0118] In addition to employing automated evaluation to verify the performance of the proposed model in extracting summaries, this invention also conducted manual evaluation to further enhance the comprehensiveness of the assessment. To this end, 100 test samples were randomly selected from the test dataset, and three graduate students fluent in English were invited to compare the summaries generated by LED, mLED, and the proposed model. During the evaluation process, this invention focused on three key indicators: information completeness (CR), fluency (FL), and conciseness (CC) of the summaries, with each indicator scored on a scale of 1 (worst) to 5 (best). The specific results of the manual evaluation are shown in Table 4.

[0119] Table 4. Human evaluation results on the Cross-SumSurvey dataset.

[0120]

[0121] The manual evaluation results in Table 4 show that the method proposed in this invention achieves a systematic breakthrough in generation quality. In terms of overall performance, the Ours model significantly outperforms the baseline model in all three metrics: information integrity, fluency, and conciseness. Its average score is 0.43 higher than mLED and 0.67 higher than LED, validating the effectiveness of the multi-stage optimization framework. Specifically, in terms of information integrity, the CR score of the proposed model is 0.36 higher than mLED, attributed to the accurate positioning of the core document propositions by the hierarchical extraction mechanism; the fluency index (FL) is 0.32 higher than mLED, reflecting the role of intermediate representation bridging technology in repairing cross-linguistic grammatical gaps; and the peak performance of conciseness (CC) at 3.92 (0.6 higher than mLED) reveals the unique advantage of the dynamic pruning algorithm in suppressing redundant information. Further analysis of the generation quality evolution path reveals that the 0.28 improvement in the CR index from LED to mLED mainly benefits from the long document modeling capability of the sparse attention mechanism; while the 0.32 gain from mLED to Ours originates from the alignment optimization of the semantic space by cross-linguistic consistency loss. In particular, the results of manual evaluation and the automatic indicators showed a high degree of consistency, which doubly verified the robustness of the method.

[0122] In summary, the results of manual evaluation and the automatic evaluation indicators corroborate each other, fully verifying the effectiveness and practicality of this invention in improving the quality of long document summarization tasks, especially in ensuring that the summary information is complete, the content is concise, and the logic is coherent.

[0123] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A cross-language long document summarization method based on dynamic latent key information constraints, characterized by: The method includes: Step 1: Construction of a cross-language long document summarization dataset: A method for constructing cross-language long document summarization based on a large language model with a prompting mechanism is proposed, and a round-trip translation strategy is introduced to construct a cross-language long document summarization dataset with high semantic consistency. Step 2: Construction of a cross-language long document summarization model based on dynamic latent key information constraints: By transforming the cross-language long document summarization task into a method for extracting dynamic key information and generating a hybrid source language summary and target language summary, the model's ability to fully model the semantic information of long documents is improved; and the source language summary is introduced as auxiliary information to enhance cross-language semantic consistency through joint decoding. Step 3: Training of cross-language long document summarization model based on dynamic potential key information constraints: Introducing a consistency loss function and achieving end-to-end joint optimization through gradient collaboration mechanism to generate more coherent and accurate cross-language long document summaries; Step 4: Generate cross-language summaries of long document content: Build a complete cross-language long document summarization prototype system; using the Sanic and Vue technology frameworks, implement a complete functional process where users submit long documents through the front-end interface, the back-end calls the model to generate summaries, and the processing results are displayed to users in an intuitive way. Step 2 includes: Step 2.1: Construct a dynamic key information extractor module: This module models sentences in long documents from the long document summary dataset constructed in Step 1 based on a pre-trained semantic model. By scoring the sentences, it selects the combination of sentences with the highest scores, thereby achieving automatic identification and extraction of key information sentences in the document. Step 2.2, Construct a cross-language summarization generator module: This module takes the source language summary as a prior condition and adopts a shared encoder architecture to simultaneously model the source language summary and the target language summary; Step 3 includes: Step 3.1: Select the sentence group with the highest ROUGE score obtained through greedy search. As a supervisory signal, the dynamic key information extractor module extracts sentence groups and... Cross-entropy loss between As part of the loss of the dynamic key information extractor module, the specific calculation formula is as follows: ； in, The parameter is The scoring function of the dynamic key information extractor module; X is the set of all fragments in the document; Indicates a monitoring signal The number of segments, For query vector, Indicates the first A text fragment; Step 3.2: Calculate the generation loss by combining the generated monolingual and cross-lingual summaries with the referenced monolingual and cross-lingual summaries. The specific calculation method is as follows: ； in, The parameter is Conditional probability distribution of the cross-language summary generator module; For the first The source language summary sequence generated step by step, For the first The target language summary sequence generated in the first step; , Indicates at time step The previously generated source language and target language summary sequences; For the maximum time step, For scalar score Sort the segments of the entire document and select the ones with the highest scores. A segment; Step 3.3: Employ a consistency alignment strategy based on dynamic weights to improve model consistency by constraining the latent space representations of different modules; calculate the probability distribution of the dynamic key information extractor module. Dynamic weights in the cross-language summary generator module KL divergence between distributions enables cross-module collaborative optimization; ； in, This indicates that the dynamic key information extractor module extracts fragments. The probability distribution of choice; This represents the distribution of dynamically weighted components in the cross-language summarization generator module. Denotes KL divergence, This represents a measure of the difference between two probability distributions. By measuring the difference between the two probability distributions, the dynamic key information extractor module and the cross-language summary generator module are encouraged to converge in decision-making.

2. The cross-language long document summarization method based on dynamic potential key information constraints according to claim 1, characterized in that: Step 1 includes: Step 1.1: Using a large language model translation method based on prompting mechanism, the English reference summaries in the SumSurvey long document summaries dataset are converted into Chinese cross-language summaries. Step 1.2: Use a machine translation system to back-translate the generated Chinese cross-language summary to obtain the corresponding English back-translated summary; Step 1.3: Assess the similarity between the English back-translated abstract and the original English reference abstract, and calculate the ROUGE-1 and ROUGE-2 scoring indicators; compare the scoring indicators with the threshold, retain the abstract pairs that meet the threshold, and discard the rest.

3. The cross-language long document summarization method based on dynamic potential key information constraints according to claim 1, characterized in that: Step 2.1 includes: Step 2.1.1: In the cross-language long document summarization task, group consecutive text segments and calculate the encoding vector for each segment separately; Step 2.1.2: Map the encoded vector to the scalar score using a Multilayer Perceptron (MLP). ,in, These are the parameters of the extractor. For query vector, Indicates the first A text fragment, The parameter is The scoring function of the dynamic key information extractor module; Step 2.1.3, Based on scalar scores Sort the segments of the entire document and select the highest-scoring ones. A fragment As input for summary generation: ； in, A collection of all fragments in the document; The preset number of sentence segments, To select the highest score A fragment.

4. The cross-language long document summarization method based on dynamic potential key information constraints according to claim 1, characterized in that: Step 2.2 includes: Step 2.2.1: Generate target summary Replace with source language summary and target language summary The sequence is connected; the specific formula is as follows: ； in, Source language summary generated for the model , Target language summary generated for the model , and These represent the start and end markers of the output summary, respectively. It is used for separation and Special markings; Step 2.2.2: Under the new generation objective, the cross-language summarization generator module will select the highest-scoring module. As input, the decoder first learns to generate Then and Generate under the conditions The specific formula is as follows: ； in, The model parameters are: Conditional probability distribution of the cross-language summary generator module; Indicates that given a query vector and Under these conditions, the model generates the entire target language summary sequence. The total probability, For the first The target language summary sequence generated in the first step; Indicates at time step The previously generated target language summary sequence; This represents the maximum time step.

5. The cross-language long document summarization method based on dynamic latent key information constraints according to claim 1, characterized in that: Step 4 includes: Step 4.1: Save the trained cross-language long document summarization model based on dynamic latent key information constraints as a ".pth" file, and load the cross-language long document summarization model based on dynamic latent key information constraints into memory using the Flask framework; Step 4.2: Build a complete cross-language long document summarization prototype system to provide efficient core information extraction and summary generation services; the core functions of the system include: long document preprocessing, summary generation based on key information extraction, cross-language summary conversion and result visualization; Step 4.3: By calling the cross-language long document summarization model based on dynamic potential key information constraints deployed on the server through the web client, the input long document is summarized, and the generated summary results are displayed on the front-end page in real time, realizing an interactive summary service for users.

6. A cross-language long document summarization device based on dynamic potential key information constraints, characterized in that: The apparatus includes a module for performing the cross-language long document summarization method based on dynamic potential key information constraints as described in any one of claims 1 to 5.

Images