Generative abstract method and system based on dual information guidance

Abstract

The embodiment of the invention provides a generative abstract method and system based on dual information guidance, and the method comprises the steps: designing a significance score according to the contribution degree of an original text sentence to an abstract, and dividing the significance score into static significance information and dynamic significance information according to the score condition; the method comprises the following steps: encoding each sentence of an original text, obtaining embedded vector representation of the sentence, predicting a significance score of each sentence through a classification model to form static significance information, and extracting first K sentences from the original text based on the significance score to form dynamic significance information; the dynamic saliency information is fused into a cross attention mechanism of the decoder through a dynamic saliency fusion module, and the static saliency information is fused into the cross attention mechanism of the decoder in an embedded form through a static saliency fusion module; and performing end-to-end training on the abstract generation model by predicting a sentence significance level and taking the sum of a loss function of sentence significance classification and a loss function of abstract generation as a total training target.

Description

Technical Field

[0001] This document relates to the field of abstract generation technology, and in particular to a generative abstracting method and system based on dual information guidance. Background Technology

[0002] Generative summarization methods reorganize language based on an understanding of the original text to generate a summary, a process that more closely resembles human workflow. However, because the decoder translates the original text into the target summary, this end-to-end process lacks guiding constraints, potentially generating unfaithful summaries containing factual errors or illusory content. Furthermore, generative methods struggle to control which information from the original text the model focuses on during summary generation—that is, which unknown aspects of the original text the generated summary should cover—leading to incomplete coverage of key information in the original text.

[0003] Currently, generative summarization methods typically use original text and corresponding human-generated summaries as training samples for supervised training. The basic idea is for the model to learn the implicit information in the original text and decide which content should be included in the summary. Recent research has proposed adding guidance to generative summarization models to deepen their understanding of the original text. Extracting guidance information from the original text is considered an effective strategy. This strategy first extracts important information and then generates a summary based on the extracted information. However, using only a single piece of guidance information is not perfect. First, key information extracted from the original text, such as keywords and sentences, is not necessarily reliable. For example, insufficient key information extracted from the original text will lead to missing key information in the generated content; conversely, too much information extracted from the original text will introduce noise and confuse the model's training. Second, existing methods extract fragments of key information from the original text. Since the distribution and quantity of key information vary depending on the document type, this is not conducive to training the model to achieve generalization ability in generating summaries. If the model extracts discrete information fragments for all types of documents, it will lead to model bias and limit the generalization ability of the generated summaries. Therefore, a good guide should fully include the key information of the original text to ensure the authenticity and completeness of the generated summary, while also enabling the model to have generalized summary generation capabilities. Summary of the Invention

[0004] This specification provides one or more embodiments of a generative summarization method guided by dual information, including: S1. Design a significance score based on the contribution of sentences in the original text to the abstract, score all sentences in the text, and divide them into static significance information and dynamic significance information according to the scores; S2. Encode each sentence in the original text, obtain the embedding vector representation of the sentence, predict the salience score of each sentence through a classification model to constitute the static salience information, and extract the first K sentences from the original text based on the salience score to constitute the dynamic salience information. S3. The dynamic saliency information is fused into the cross-attention mechanism of the decoder through the dynamic saliency fusion module, and the static saliency information is fused into the cross-attention mechanism of the decoder in an embedded form through the static saliency fusion module; S4. By predicting the saliency level of sentences, the sum of the loss function for sentence saliency classification and the loss function for summary generation is used as the overall training objective to train the summary generation model end-to-end.

[0005] Furthermore, in step S1: The saliency score is divided into multiple levels based on the similarity between the sentence and the real summary, and the sentence receives a saliency score according to its level. The saliency distribution of sentences in the entire text is defined as static saliency information, while the text composed of sentences with high contribution is defined as dynamic saliency information.

[0006] Furthermore, obtaining the sentence's embedding vector representation specifically involves: A special token is added after the token sequence of each sentence in the original text. After the sentence is input into the encoder, the hidden state of the special token in the last layer of the encoder is taken as the embedding vector representation of the sentence.

[0007] Furthermore, the saliency score for each sentence is predicted using a classification model as follows: The embedding vector representation of the sentence is input into a multilayer perceptron, which maps it to a logistic value vector. The probability of the sentence belonging to each saliency level is then calculated using the softmax function, and the level with the highest probability is taken as the saliency score of the sentence.

[0008] Furthermore, the dynamic saliency fusion module is specifically used for: The dynamic saliency information is encoded using an encoder that encodes the original text to obtain its initial embedding representation; The initial embedding representation is input into a difference capture module to capture the high-level semantic differences between the dynamic saliency information and the original text, thereby obtaining the difference-enhanced dynamic saliency embedding representation; A first cross-attention layer is added to the decoder, and the dynamic saliency embedding representation after difference enhancement is used as the key and value of the attention layer.

[0009] Furthermore, the static saliency fusion module is specifically used for: The static saliency information is mapped to a static saliency embedding representation; The static saliency embedding representation is added to the token embedding representation of the original text to form the fused original text embedding representation; In the decoder, the fused original text embedding representation is used as the key and value of the second cross-attention layer.

[0010] Furthermore, during prediction, the average cross-entropy loss of all sentences in the document is used. As a classification loss: ; The model uses negative log-likelihood loss during summary generation. : ; Adding these two functions together gives the final training loss. :

[0011] in , It is the balance coefficient of these two loss functions.

[0012] This specification provides one or more embodiments of a generative summarization system based on dual-information guidance, characterized in that it includes: The saliency information definition module is used to design saliency scores based on the contribution of sentences in the original text to the abstract, score sentences in the whole text, and divide them into static saliency information and dynamic saliency information according to the scores. The saliency information acquisition module is used to encode each sentence in the original text, obtain the embedding vector representation of the sentence, predict the saliency score of each sentence through a classification model to form the static saliency information, and extract the first K sentences from the original text based on the saliency score to form the dynamic saliency information. The saliency information fusion module is used to fuse the dynamic saliency information into the cross-attention mechanism of the decoder through the dynamic saliency fusion module, and to fuse the static saliency information into the cross-attention mechanism of the decoder in an embedded form through the static saliency fusion module. Model training module: This module is used to train the summarization model end-to-end by using the sum of the loss function for sentence saliency classification and the loss function for summarization generation as the overall training objective, based on the prediction of sentence saliency levels.

[0013] This specification provides one or more embodiments of an electronic device, including: Processor; and, A memory is configured to store computer-executable instructions, which, when executed, cause the processor to implement the steps of the above-described generative summarization method based on dual information guidance.

[0014] This specification provides one or more embodiments of a storage medium for storing computer-executable instructions that, when executed, implement the steps of the above-described generative summarization method based on dual information guidance.

[0015] By employing the embodiments of the present invention, the problem of difficulty in constraining which content of the original text should be included in the generated summary can be alleviated to a limited extent. Based on the pre-trained model, the generative summarization task is modeled as a unified architecture that integrates static and dynamic saliency information: the global saliency distribution of the original text is calculated as static saliency information, and key sentences are extracted from the original text to form dynamic saliency information. These together guide the summary generation process, thereby solving the problem of model constraint and improving the performance of the generated summary.

[0016] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in one or more embodiments of this specification or in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating a dual-information-guided generative summarization method provided for one or more embodiments of this specification; Figure 2 A schematic diagram of the overall framework of a generative summarization method based on dual information guidance provided for one or more embodiments of this specification; Figure 3 A schematic diagram illustrating saliency prediction for a generative summarization method based on dual information guidance, provided for one or more embodiments of this specification; Figure 4 This is a schematic diagram illustrating the observation time results of the importance distribution of the WikiHow dataset in one embodiment of this specification; Figure 5A schematic diagram illustrating the composition of a generative summarization system based on dual-information guidance, provided for one or more embodiments of this specification; Figure 6 This is a schematic diagram of the structure of an electronic device provided for one or more embodiments of this specification. Detailed Implementation

[0019] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this document.

[0020] Method Implementation Examples According to embodiments of the present invention, a generative summarization method based on dual information guidance is provided. The overall framework of the present invention is as follows: Figure 2 As shown, a Transformer-based Encoder-Decoder structure is used for summarization, encapsulating static saliency information, dynamic saliency information extraction, and text summarization into the same network, and performing end-to-end training and learning through forward propagation. During training, the model learns to predict the saliency score of each sentence based on the true saliency score based on ROUGE, and then extracts sentence saliency information based on the sentence saliency scores to guide the generation of the summary. During the inference phase, the model first predicts the saliency score of each sentence in the original text to form static guidance information, then extracts highly saliency sentences from the original text based on the scores to form dynamic saliency information, and finally uses the static and dynamic information of the text to guide the generation of the summary.

[0021] Figure 1 A flowchart illustrating a dual-information-guided generative summarization method provided for one or more embodiments of this specification is shown below. Figure 1 As shown, the generative summarization method based on dual information guidance according to an embodiment of the present invention specifically includes: S1. Design a significance score based on the contribution of sentences in the original text to the abstract, score all sentences in the text, and divide them into static significance information and dynamic significance information according to the scores.

[0022] First, define the significance score, given... For have The source text sequence of each token For having A digest sequence of tokens, where and All in the vocabulary list In China. Representing the The saliency scores of each sentence, among which L is the number of sentences in the source text. Definition For the first The sentence index of each token, where .

[0023] The saliency score is divided into multiple levels based on the similarity between sentences and the real summary, and sentences receive a saliency score according to their respective levels. The saliency distribution of sentences throughout the text is defined as static saliency information, while the text composed of sentences with high contribution is defined as dynamic saliency information. Specifically, the saliency score of each token is obtained from the saliency score of its corresponding sentence, defined as... ,in This represents a function that maps a token to an embedding vector. This is static significance information. Defined as dynamic saliency information obtained based on the saliency score of the original sentence.

[0024] The summary generation task is defined as follows: ; The generation of each digest token is done from front to back. Tokens and sequences generated step by step , original text sequence Dynamic saliency information and the static significance information of the original text. Decide.

[0025] S2. Encode each sentence in the original text to obtain the embedding vector representation of the sentence, predict the saliency score of each sentence through a classification model to constitute the static saliency information, and extract the first K sentences from the original text based on the saliency score to constitute the dynamic saliency information.

[0026] like Figure 3 As shown, in order to obtain the saliency score of each sentence in the original text, this invention adds a special token after the token sequence of each sentence in the original text. After inputting the sentence into the encoder, the hidden state of the special token in the last layer of the encoder is taken as the embedding vector representation of the sentence. ; ; For sentences in the original text, they are divided into multiple saliency categories, based on the obtained... Predicting which sentence belongs to a certain category probability This category is used as the saliency score of the sentence. A multilayer perceptron (MLP) is used to process each embedding vector. Mapped to a logical value vector , Defined as: ; Then, the probability of the sentence belonging to each saliency level is calculated using the softmax function: ; The salience score of the sentence is the rank with the highest probability.

[0027] The saliency scores of all sentences in the original text obtained from the above calculation Obtain static significance information, based on Extract the top with the highest significance level Sentences constitute salient text, serving as dynamic salient information to guide the model: ; In the stage of acquiring dynamic saliency information, since the number of highly significant sentences in each sample is different, the number of sentences extracted by the method of this invention is... It is not a fixed value, but is determined by the sample itself. The method of this invention extracts the sentences with the highest significance level, so that the number of significant sentences extracted varies from sample to sample.

[0028] S3. The dynamic saliency information is fused into the cross-attention mechanism of the decoder through the dynamic saliency fusion module, and the static saliency information is fused into the cross-attention mechanism of the decoder in an embedded form through the static saliency fusion module.

[0029] The dynamic saliency information is fused into the cross-attention mechanism of the decoder through the dynamic saliency fusion module as follows: The dynamic saliency information is encoded using an encoder that encodes the original text to obtain its initial embedding representation. Important sentences from the original text are then extracted to form the dynamic saliency information. This serves as guiding information, explicitly indicating the important content of the original model text, in order to obtain dynamic saliency information. Then, use the encoded original text. Encoder to encode To obtain its embedding representation .

[0030] The initial embedding representation is input into a difference-capturing module to capture high-level semantic differences between the dynamic saliency information and the original text, resulting in a difference-enhanced dynamic saliency embedding representation. To capture the high-level differences between the original text and the guiding dynamic saliency information, a difference-capturing module is added to the saliency text after the native encoder. The difference between the dynamic saliency information captured at the top level and the original text is as follows: ; ; in, It is a salient text embedding vector after differential differentiation.

[0031] To enable the model to receive this hint, a first cross-attention layer is added to the decoder, and the dynamic saliency embedding representation after difference enhancement is used as the key and value of this attention layer.

[0032] The cross-attention module of the first cross-attention layer is represented as follows: ; Cross attention , It is the hidden state of dynamic saliency information after difference capture. , It is the hidden state of the self-attention layer in the decoder.

[0033] The static saliency information is fused into the cross-attention mechanism of the decoder in an embedded form through the static saliency fusion module, specifically as follows: The static significance information of the original text Integrating this into the decoder's cross-attention mechanism prompts the model with information about the distribution of important content from the original text, enabling the model to focus on implicit saliency information at a global level. Specifically, firstly... Mapped to a static saliency embedding representation for training: ; During the training phase, the static information used consists of the distribution of the true saliency scores of the original text, which can therefore be directly mapped to the embedding representation; during the inference and generation phase, the model predicts at the level of each input sentence. probability Therefore, the level with the highest probability should be used for reasoning: ; get After encoding the embedding representation, the static saliency embedding representation is added to the original text's token embedding representation to form a fused original text embedding representation; the added token embedding is then used as... The status, specifically described as: ; ; in This is the embedded representation of the original text, and cross-attention is expressed as: ; in It is a plaintext embedding without fusion of static saliency. , This is the hidden state after the decoder passes through SSF. In the decoder, the fused original text embedding representation is used as the key and value of the second cross-attention layer.

[0034] S4. By predicting the saliency level of sentences, the sum of the loss function for sentence saliency classification and the loss function for summary generation is used as the overall training objective to train the summary generation model end-to-end.

[0035] During training, the model predicts the saliency level for each sentence and uses this level as the saliency score for that sentence. During prediction, the average cross-entropy loss across all sentences in the document is used. As a classification loss: ; The model uses negative log-likelihood loss during summary generation. : ; Adding these two functions together gives the final training loss. :

[0036] in , It is the balance coefficient of these two loss functions.

[0037] This invention was tested on three summary datasets. For each dataset, the original text of the dataset was used as input, and its corresponding summary was used as the actual output. XSum is a partially generative dataset containing one-sentence summaries of BBC online articles, with 203028 / 11273 / 11332 training / validation / test data pairs. CNN / DM consists of news articles from CNN and the Daily Mail website and their manually written summaries, containing 287084 / 13367 / 11489 training / validation / test data pairs. WikiHow is a dataset extracted from an online knowledge base with high abstraction, containing 167932 / 6000 / 6000 training / validation / test data pairs.

[0038] This invention demonstrates the evaluation results of the widely used ROUGE index (F1), including word-level ROUGE-1 (R1), ROUGE-2 (R2), and sentence-level ROUGE-L (RL). This invention uses ROUGE-1.5.5 to calculate the ROUGE results. This invention compares its method with mainstream approaches for similar tasks, including extractive, generative, and hybrid baseline models: LEAD-3 is a common extractive summarization method that extracts the first three sentences of the original text as a document summary. BertExt is an extractive model whose parameters are initialized by Bert; BertAbs is an extractive model where the encoder is initialized with Bert, and the decoder is trained using an optimizer different from the encoder; MatchSum is an extractive model that reorders candidate summaries generated by BertExt and achieves state-of-the-art extraction results on various summarization datasets; Bart is a state-of-the-art abstract summarization model pre-trained with a denoised autoencoding target. CIT uses a keyword extractor and selective encoding mechanism to guide summary generation. GSum uses the summaries extracted by MatchSum as guidance to supervise Bart in abstract summarization. SEASON modifies the Bart architecture by using the encoder to predict the document's saliency score to guide the decoder in generating summaries. Bart w / Renc+dec uses the encoder to predict the summary length and fine-tunes it in the decoder to enable the model to understand specific information.

[0039] To ensure fairness in the comparative experiments, this invention uses the same implementation details as the baseline, fine-tuning the Bart-large model on the three experimental datasets. To conserve computational resources, this invention uses the same encoder to encode both the original text and the extracted salient text, while adding a separate encoder layer after the encoded salient text to produce a high level of distinction. For salient prediction, this invention uses a single-layer MLP implementation containing a linear layer. For all three datasets, this invention truncates the input sequence to 1024 tokens (including a special token added after each sentence) to accommodate Bart's maximum input length. Since the reference summary length distribution differs across the four datasets, this invention sets the summary lengths for XSum, CNN / DM, and WikiHow to 60 / 142 / 140 respectively to ensure that over 99% of the reference summaries are preserved.

[0040] Experimental Results Analysis The experimental results of this invention are shown in Table 1 and Table 2: Table 1

[0041] Table 2

[0042] On the WikiHow dataset, the proposed method improves ROUGE1 / 2 / L scores by 4.14 / 2.39 / 4.29 compared to the original Bart model, demonstrating that the proposed method can more effectively guide the model to improve summary quality. Compared to Bart w / Renc+dec, which uses predicted summary length information to guide summary generation, the proposed method uses predicted importance distribution information of the original text to guide summary generation, achieving better performance improvement. On the CNN / DM dataset, the proposed method improves ROUGE1 / 2 / L scores by 2.42 / 2.13 / 2.3 compared to the original Bart model, and by 0.31 / 0.77 / 0.12 compared to SEASON, demonstrating the effectiveness of using dynamic saliency information as guidance. Results on the XSum dataset further verify that the proposed method can achieve improvements on different datasets, achieving improvements of 0.89 / 0.74 / 0.43 in ROUGE1 / 2 / L scores compared to the Bart model. The superior performance of the method in this invention is attributed to two aspects: concretizing the static saliency information of the original text, enabling the model to more clearly perceive the important information in the original text, and extracting highly saliency sentences from the original text as dynamic saliency information to prompt the model to generate a summary.

[0043] This invention presents ablation studies on the static saliency fusion module (SSF) and dynamic saliency fusion module (DSF) in the proposed method, as shown in Table 3: Table 3

[0044] The ablation experiments were conducted on a model trained on the WikiHow dataset. "w / o DSF" means removing dynamic saliency information and using only static saliency information to guide the model, while "w / o SSF" does the opposite.

[0045] As shown in Table 3, after removing the Dynamic Saliency Fusion Module (DSF) from the model, ROUGE1 / 2 / L decreased by 0.79 / 0.73 / 0.67 respectively, resulting in a significant performance drop. This indicates that the module makes a positive contribution to model inference, and incorporating dynamic saliency into the decoder as a guide is beneficial. Similarly, after removing the Static Saliency Fusion Module (SSF), this invention found that the model's performance on the WikiHow test set also decreased significantly, with ROUGE1 / 2 / L decreasing by 2.35 / 1.21 / 2.08 respectively. This suggests that the model performs better after learning the importance distribution information of sentences in the original text.

[0046] To further analyze the impact of SSF, this invention conducted an observational experiment on the importance distribution of the WikiHow dataset, and the results are as follows: Figure 4 As shown.

[0047] This invention obtains the above results using Algorithm 1. The X-axis represents the position of a sentence in the original text; for example, 10 represents a position in the top 10%, 20 represents a position between 10% and 20%, and so on. The Y-axis represents the proportion of sentences at the corresponding position to the total number of sentences in the original text. The red curve represents the distribution of sentences with the highest importance level in the original text. As can be seen from the graph, sentences with the highest importance level are distributed in various positions in the original text, making it difficult for the generation model to know the importance distribution of the original text. This invention uses the importance distribution of the original text as a static saliency guide for the model to generate summaries. Based on the prompts, the model can better obtain the importance distribution of the original text, thus achieving a better performance improvement.

[0048] The beneficial effects of this invention are as follows: This approach can alleviate the problem of generating summaries struggling to constrain which parts of the original text should be included. Based on a pre-trained model, the generative summarization task is modeled as a unified architecture that integrates static and dynamic saliency information: the global saliency distribution of the original text is calculated as static saliency information, and key sentences are extracted from the original text to form dynamic saliency information. These two aspects work together to guide the summarization process, thereby solving the problem of model constraints and improving the performance of the generated summaries.

[0049] System Implementation Examples According to embodiments of the present invention, a generative summarization system based on dual information guidance is provided. Figure 5 A schematic diagram illustrating the composition of a generative summarization system based on dual-information guidance, provided for one or more embodiments of this specification, is shown below. Figure 5 As shown, the generative summarization system based on dual information guidance according to an embodiment of the present invention specifically includes: The saliency information definition module 50 is used to design saliency scores based on the contribution of sentences in the original text to the abstract, to score sentences in the whole text, and to divide them into static saliency information and dynamic saliency information according to the scores. The saliency information acquisition module 52 is used to encode each sentence in the original text, obtain the embedding vector representation of the sentence, predict the saliency score of each sentence through a classification model to constitute the static saliency information, and extract the first K sentences from the original text based on the saliency score to constitute the dynamic saliency information. Saliency information fusion module 54: used to fuse the dynamic saliency information into the cross-attention mechanism of the decoder through the dynamic saliency fusion module, and to fuse the static saliency information into the cross-attention mechanism of the decoder in an embedded form through the static saliency fusion module; Model Training Module 56: This module is used to train the summarization model end-to-end by using the sum of the loss function for sentence saliency classification and the loss function for summarization generation as the overall training objective, based on the prediction of sentence saliency levels.

[0050] The embodiments of the present invention are system embodiments corresponding to the above method embodiments. The specific operation of each module can be understood by referring to the description of the method embodiments, and will not be repeated here.

[0051] Device Example 1 This invention provides an electronic device, such as... Figure 6 As shown, it includes: a memory 60, a processor 62, and a computer program stored in the memory 60 and executable on the processor 62. When the computer program is executed by the processor 62, it performs the following method steps: S1. Design a significance score based on the contribution of sentences in the original text to the abstract, score all sentences in the text, and divide them into static significance information and dynamic significance information according to the scores; S2. Encode each sentence in the original text, obtain the embedding vector representation of the sentence, predict the salience score of each sentence through a classification model to constitute the static salience information, and extract the first K sentences from the original text based on the salience score to constitute the dynamic salience information. S3. The dynamic saliency information is fused into the cross-attention mechanism of the decoder through the dynamic saliency fusion module, and the static saliency information is fused into the cross-attention mechanism of the decoder in an embedded form through the static saliency fusion module; S4. By predicting the saliency level of sentences, the sum of the loss function for sentence saliency classification and the loss function for summary generation is used as the overall training objective to train the summary generation model end-to-end.

[0052] Device Example 2 This invention provides a computer-readable storage medium storing an information transmission implementation program. When executed by a processor 62, the program performs the following method steps: S1. Design a significance score based on the contribution of sentences in the original text to the abstract, score all sentences in the text, and divide them into static significance information and dynamic significance information according to the scores; S2. Encode each sentence in the original text, obtain the embedding vector representation of the sentence, predict the salience score of each sentence through a classification model to constitute the static salience information, and extract the first K sentences from the original text based on the salience score to constitute the dynamic salience information. S3. The dynamic saliency information is fused into the cross-attention mechanism of the decoder through the dynamic saliency fusion module, and the static saliency information is fused into the cross-attention mechanism of the decoder in an embedded form through the static saliency fusion module; S4. By predicting the saliency level of sentences, the sum of the loss function for sentence saliency classification and the loss function for summary generation is used as the overall training objective to train the summary generation model end-to-end.

[0053] The computer-readable storage media described in this embodiment include, but are not limited to, ROM, RAM, disk, or optical disk.

[0054] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A generative summarization method based on dual-information guidance, characterized in that, include: S1. Design a significance score based on the contribution of sentences in the original text to the abstract, score all sentences in the text, and divide them into static significance information and dynamic significance information according to the scores; S2. Encode each sentence in the original text, obtain the embedding vector representation of the sentence, predict the salience score of each sentence through a classification model to constitute the static salience information, and extract the first K sentences from the original text based on the salience score to constitute the dynamic salience information. S3. The dynamic saliency information is fused into the cross-attention mechanism of the decoder through the dynamic saliency fusion module, and the static saliency information is fused into the cross-attention mechanism of the decoder in an embedded form through the static saliency fusion module; S4. By predicting the saliency level of sentences, the sum of the loss function for sentence saliency classification and the loss function for summary generation is used as the overall training objective to train the summary generation model end-to-end.

2. The method according to claim 1, characterized in that, In step S1: The saliency score is divided into multiple levels based on the similarity between the sentence and the real summary, and the sentence receives a saliency score according to its level. The saliency distribution of sentences in the entire text is defined as static saliency information, while the text composed of sentences with high contribution is defined as dynamic saliency information.

3. The method according to claim 1, characterized in that, The specific steps to obtain the sentence's embedding vector representation are as follows: A special token is added after the token sequence of each sentence in the original text. After the sentence is input into the encoder, the hidden state of the special token in the last layer of the encoder is taken as the embedding vector representation of the sentence.

4. The method according to claim 1, characterized in that, The saliency score of each sentence is predicted using a classification model as follows: The embedding vector representation of the sentence is input into a multilayer perceptron, which maps it to a logistic value vector. The probability of the sentence belonging to each saliency level is then calculated using the softmax function, and the level with the highest probability is taken as the saliency score of the sentence.

5. The method according to claim 1, characterized in that, The dynamic saliency fusion module is specifically used for: The dynamic saliency information is encoded using an encoder that encodes the original text to obtain its initial embedding representation; The initial embedding representation is input into a difference capture module to capture the high-level semantic differences between the dynamic saliency information and the original text, thereby obtaining the difference-enhanced dynamic saliency embedding representation; A first cross-attention layer is added to the decoder, and the dynamic saliency embedding representation after difference enhancement is used as the key and value of the attention layer.

6. The method according to claim 1, characterized in that, The static saliency fusion module is specifically used for: The static saliency information is mapped to a static saliency embedding representation; The static saliency embedding representation is added to the token embedding representation of the original text to form the fused original text embedding representation; In the decoder, the fused original text embedding representation is used as the key and value of the second cross-attention layer.

7. The method according to claim 1, characterized in that, During prediction, the average cross-entropy loss of all sentences in the document is used. As a classification loss: ； The model uses negative log-likelihood loss during summary generation. : ； Adding these two functions together gives the final training loss. : in , It is the balance coefficient of these two loss functions.

8. A generative summarization system based on dual-information guidance, characterized in that, include: The saliency information definition module is used to design saliency scores based on the contribution of sentences in the original text to the abstract, score sentences in the whole text, and divide them into static saliency information and dynamic saliency information according to the scores. The saliency information acquisition module is used to encode each sentence in the original text, obtain the embedding vector representation of the sentence, predict the saliency score of each sentence through a classification model to form the static saliency information, and extract the first K sentences from the original text based on the saliency score to form the dynamic saliency information. The saliency information fusion module is used to fuse the dynamic saliency information into the cross-attention mechanism of the decoder through the dynamic saliency fusion module, and to fuse the static saliency information into the cross-attention mechanism of the decoder in an embedded form through the static saliency fusion module. Model training module: This module is used to train the summarization model end-to-end by using the sum of the loss function for sentence saliency classification and the loss function for summarization generation as the overall training objective, based on the prediction of sentence saliency levels.

9. An electronic device, characterized in that, include: processor; as well as, A memory configured to store computer-executable instructions, which, when executed, cause the processor to implement the steps of the generative summarization method based on dual information guidance as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, Used to store computer-executable instructions, which, when executed, implement the steps of the generative summarization method based on dual information guidance as described in any one of claims 1 to 7.

Images