A two-stage text summarization method based on PEGASUS model and dynamic error correction

Abstract

The present invention discloses a two-stage text summarization method based on the PEGASUS model and dynamic error correction. The method obtains text data to be processed, performs preprocessing, obtains various features, fuses them, and obtains sentence weights. Sentences are selected according to the sentence weights to select key sentences for the summary. Hierarchical clustering is performed on the key sentences. Potential unregistered words are identified, marked as incorrect words, and replaced with alternative words, and the alternative words are corrected by substitution loss. The replacement text is subjected to the PEGASUS model to calculate the contrast loss. The comprehensive loss is calculated by the substitution loss and the contrast loss, and the weight coefficients, dynamic thresholds, and similarity thresholds are adjusted until the comprehensive loss reaches a minimum or the number of iterations reaches a maximum, and then a summary is output. The unregistered words are replaced with different alternative words by the "dynamic error correction mechanism" to achieve the optimal selection of the summary and improve the accuracy of the generated summary.

Description

Technical Field

[0001] The present invention relates to text summary generation, and in particular to a two-stage text summary generation method based on a PEGASUS model and dynamic error correction. Background Art

[0002] The two-stage text summarization task primarily involves the following four technical aspects: First, key sentence extraction, a fundamental task in two-stage summarization, aims to extract important, information-dense sentences from the original text. Commonly used techniques include graph-based ranking algorithms such as TextRank, deep learning methods such as BERT embedding similarity comparison, and machine learning methods such as Support Vector Machines (SVMs). These methods focus on identifying and extracting the most relevant sentences to the topic and excel in information extraction. Second, text encoding and understanding also play a crucial role in the two-stage task, particularly in deep semantic understanding. Models such as BERT and Transformer are widely used in this stage, effectively capturing contextual information and improving the model's ability to understand the text's intent and semantics. Third, abstractive summary generation is the second crucial step in the two-stage process. By using generative pre-trained models such as PEGASUS or T5, the model can semantically reorganize and simplify the extracted key sentences, generating a fluent and concise summary. This stage emphasizes generating natural language text while preserving core information. Fourth, model optimization and integration are crucial for ensuring the quality of summary generation. By using techniques such as knowledge distillation, adversarial training, and multi-task learning, the generalization performance and generation accuracy of the model can be improved.

[0003] The main problems in current tasks are as follows: difficulty in sentence selection due to the diversity of text topics; the balance between the naturalness and accuracy of generated summaries; the existence of unregistered words and exposure bias; and computational efficiency issues when processing long texts. Summary of the Invention

[0004] Purpose of the invention: To address the above shortcomings, the present invention provides a highly accurate two-stage text summarization method based on the PEGASUS model and dynamic error correction.

[0005] Technical solution: To solve the above problems, the present invention adopts a two-stage text summarization method based on the PEGASUS model and dynamic error correction, which includes the following steps:

[0006] Step 1: Obtain the text data to be processed and preprocess it to obtain the title, sentence vector, word vector, and title word vector;

[0007] Step 2: Introduce the title word vector and perform weighted averaging to calculate the weight of the word vector. Select keywords based on the weight of the word vector to obtain keyword features. Calculate the similarity between the sentence vector and the title to obtain title features. Assign different weights to sentence vectors at different positions to obtain sentence position features. Process the sentence vector through a neural network to output the final semantic features.

[0008] Step 3: Calculate the similarity between sentences to obtain a sentence similarity matrix; fuse the sentence similarity matrix, keyword features, title features, sentence position features, and final semantic features to obtain sentence feature weights;

[0009] Step 4: Select sentences based on sentence feature weights to obtain candidate key sentences; calculate the MMR value of the candidate key sentences using the MMR algorithm, add the candidate key sentences with the highest MMR value to the key sentence set, and iterate the calculation until the sentence set meets the key sentence length or number requirements, and then obtain the key sentences;

[0010] Step 5: The text data consists of several text units. The key sentences are embedded in the context using the Word2Vec model. The distance between each text unit is calculated, and the text units are hierarchically clustered based on the distance between the text units.

[0011] Step 6: Calculate the similarity of each cluster center to identify potential unregistered words, mark the unregistered words as incorrect words, generate contextual embedding representations of potential alternative words for the incorrectly marked words, and calculate the similarity between the alternative words and the incorrectly marked words based on the contextual embedding representations of the alternative words. When the similarity is higher than the threshold, replace the incorrectly marked words with the alternative words to obtain the replacement text, calculate the replacement loss of the replacement text, and correct the replacement words using the replacement loss.

[0012] Step 7: Input the replacement text into the trained PEGASUS model and output the summary;

[0013] The PEGASUS model training process is as follows: The training data includes the replacement texts of different replacement words processed from the text data and the corresponding reference summaries of the text data. The replacement texts in the training data are input into the PEGASUS model, and the generated summaries are output using a beam search strategy in both supervised and unsupervised conditions. The similarity between the reference summary and the generated summary in the supervised condition and the similarity between the generated summaries after different replacement words in the unsupervised condition are calculated, and the overall similarity is recorded as the contrastive loss.

[0014] The comprehensive loss is calculated by replacing the loss and comparing the loss, and the weight coefficients, dynamic thresholds and similarity thresholds are adjusted until the comprehensive loss reaches the minimum or the number of iterations reaches the maximum, thus obtaining a trained PEGASUS model.

[0015] Furthermore, the preprocessing in step 1 includes: removing stop words, segmenting words, and marking parts of speech on the acquired text data to obtain a text dataset; using the text dataset as input, using the sentence vector generated by the BERT model; and using the Word2Vec model to train external data to generate word vectors.

[0016] Furthermore, in step 2, the weight calculation formula of the word vector is:

[0017] W(w)=αTF_IDF(w,d)+β||v w ||+γTR(w)+δI T (w)

[0018] Among them, W(w) represents the keyword feature weight, ||v w ||Word vector v representing word w w The modulus length, TR(w) indicates the importance of word w calculated by the TextRank algorithm, I T (w) represents the judgment parameter of whether word w belongs to the title word vector, α, β, λ, δ are weight coefficients; TF_IDF(w,d) represents the frequency and distribution of word w in the text data, and the calculation formula is:

[0019]

[0020] Where TF_IDF(w,d) represents the frequency and distribution of word w in the text data, tf(w,d) represents the frequency of word w in sentence S in document d, df(w) represents the number of documents in which word w appears in all documents, and N is the total number of all documents.

[0021] The calculation formula for the title feature W(St) obtained based on the similarity between the sentence vector and the title is:

[0022] W(St)=A·sim(S,title)+B·||v s ||

[0023] Among them, sim(S, title) represents the cosine similarity between sentence S and title title, v s It is represented as a vector representation of sentence S, where A and B are weight coefficients;

[0024] The calculation formula of sentence position feature is:

[0025]

[0026] Among them, n is the number of paragraphs in the article, x is the number of sentences in the first paragraph, y is the number of sentences in the last paragraph, i and j are the positions of the sentences in the first and last paragraphs, e1, e2, β1, and β2 are weight coefficients, and p represents the parameters of different sentence positions.

[0027] Furthermore, in step 2, the sentence vector is processed by a bidirectional multi-layer GRU neural network to output the final semantic features, specifically: the sentence vector is input into the bidirectional multi-layer GRU neural network and converted into an ordered sequence x i =(x i,1 ,x i,2 ,...,x i,d ), the sequence length is d, the last hidden state of the bidirectional multi-layer GRU neural network is processed to obtain the final semantic feature h of the sentence i,fin :

[0028]

[0029] in, and Represents the last layer output of the forward and reverse hidden state vectors.

[0030] Furthermore, the calculation formula of the sentence feature weight W in step 3 is:

[0031] W=μW ConSim +c||W(w)||·h i,fin +dW(St)+qW loc (S)+ρh i,fin

[0032] Among them, W ConSim is the similarity matrix between sentences, μ, c, d, q, ρ are weight coefficients.

[0033] Furthermore, in step 4, the sentences are sorted in descending order according to the sentence feature weights, and sentences are selected as candidate key sentences according to the ratio. The MMR algorithm is used to calculate the relevance and redundancy of the candidate key sentences to obtain the MMR value of the candidate key sentences. The calculation formula is:

[0034]

[0035] Among them, MMR(s) is the MMR value of the candidate key sentence s, D is the target summary, is a set of candidate key sentences, Sim(s,D) represents the similarity between the candidate key sentence s and the target summary D, Sim(s,s') represents the maximum similarity between the candidate key sentence s and all other candidate key sentences s', and η is the weight coefficient.

[0036] Furthermore, in step 6, the replacement loss between the replacement text and the key sentences of the original text is calculated using the following formula:

[0037]

[0038] Among them, E is the key sentence of the original text, To replace the text, e i is the i-th original word in the key sentence of the original text, is the replacement word for the i-th original word, Loss function, m is the number of alternative words.

[0039] Furthermore, the similarity between the reference summary and the generated summary in the supervised case in step 7 includes semantic similarity and structural similarity;

[0040] Semantic similarity is expressed by cosine similarity, and the calculation formula is:

[0041] L sc (g,r)=1-cos(f g ,f r )

[0042] Among them, g represents the generated summary, r represents the reference summary; f g represents the embedding representation of the generated summary, f r Embedding representation of the reference summary;

[0043] The formula for calculating structural similarity is:

[0044] L st (g,r)=||S(f g )-S(f r )||

[0045] Among them, S(f g ) represents the structural feature function of the embedding representation of the generated summary, S(f r ) represents the structural feature function of the embedding representation of the reference summary;

[0046] The formula for calculating the similarity between summaries after replacing words in an unsupervised situation is:

[0047]

[0048] Among them, g j represents the unsupervised generation instance of the j-th replacement word, and M represents the number of generated instances;

[0049] The contrast loss is:

[0050] Loss2(g,r)=tL sc (g,r)+kL st(g,r)+γL con (g)

[0051] Among them, t, k, and γ are weight coefficients;

[0052] The comprehensive loss is:

[0053]

[0054] Among them, θ is the weight coefficient.

[0055] The present invention also adopts a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0056] The present invention also adopts a computer-readable storage medium having a computer program stored thereon, and the computer program implements the steps of the above method when executed by a processor.

[0057] Beneficial effects: Compared with the existing technology, the significant advantages of the present invention are that it constructs a dynamic error correction mechanism module, dynamically applies hierarchical clustering, effectively replaces unregistered words with different alternative words, and realizes the optimal selection of the summary; the present invention adopts both unsupervised and supervised methods to generate summaries, constructs a comparative learning framework, and ensures the accuracy and naturalness of the output summary; the present invention uses replacement loss and contrast loss as comprehensive losses to train the model, and can obtain summary texts with accurate expression, fluent sentences, and in line with human reading habits; the present invention uses a bidirectional multi-layer GRU neural network to extract features of key sentences and integrate them into semantic features, with few model parameters, high efficiency and good effect; the present invention divides text summary generation into two stages, and uses the extraction stage to extract and integrate key sentences, optimizes the text structure, reduces the relative input of the abstract stage, and improves the abstract efficiency and the accuracy of summary generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a flowchart of the abstract generation method of the present invention.

[0059] Figure 2 This is the first stage model framework diagram of the present invention.

[0060] Figure 3 This is the second stage model framework diagram of the present invention. DETAILED DESCRIPTION

[0061] like Figure 1 and Figure 2 As shown, in this embodiment, a two-stage text summary generation method based on the PEGASUS model and dynamic error correction includes the following steps:

[0062] Step 1: Obtain the text data to be processed, perform preprocessing on the obtained text data by removing stop words, segmenting words, and marking parts of speech to obtain a text dataset and title. Use the BERT model to process the text dataset to generate sentence vectors. Use the Word2Vec model to train the text dataset and title with external data to generate word vectors and title word vectors. Use the TextRank algorithm to construct a weighted undirected graph to obtain word vector weights, and introduce the title word vector for weighted averaging to output keyword features. The specific process of keyword feature acquisition includes:

[0063] Step 1-1: pre-process the acquired text data, remove stop words, segment words, mark parts of speech, and divide the data set into training set, validation set, and test set;

[0064] Step 1-2, use the sentence vector generated by the BERT model to perform operations and retain the overall information of the sentence. i The sentence vector E(S) is obtained through the BERT model i ). The Word2Vec model is used to generate word vectors and title vectors by introducing a large amount of external data to enrich the semantic features of the text.

[0065] In steps 1-3, we use the TextRank algorithm to construct a weighted undirected graph to obtain word vector weights, and then introduce the title word vectors for weighted averaging. For each word w in sentence S, the frequency of occurrence in document d is tf(w,d), and the number of documents in which word w appears in document set D is df(w). Document set D includes all documents. The example formula for TF_IDF of the frequency and distribution of word w is as follows:

[0066]

[0067] Where N is the total number of documents.

[0068] The title word vector is introduced to optimize the calculation of word vector weights. The example formula for the corresponding operation is as follows:

[0069]

[0070] Calculate the word vector weight. The example formula for the corresponding operation is as follows:

[0071] W(w)=αTF_IDF(w,d)+βv w ||+γTR(w)+δI T (w)

[0072] Among them, W(w) represents the word vector weight, ||v w ||Word vector v representing word w w The modulus length, TR(w) indicates the importance of word w calculated by the TextRank algorithm, IT (w) represents the judgment parameter of whether word w belongs to the title word vector, α, β, λ, δ are weight coefficients;

[0073] According to the word vector weight, the word with the highest weight is selected as the keyword. The word vector weight of the keyword is the keyword feature weight, and the keyword feature weight is used as the keyword feature;

[0074] Step 2: Extract relevant sentence features, measure the relationship between the sentence vector and the title, assign different weights to sentences in different positions, deeply understand the meaning of sentences and obtain the semantic relationship of text. The specific process of extracting sentence-related features includes:

[0075] Step 2-1: Measure the similarity between the sentence vector of the text and the title, and output the title features. The example formula for the corresponding operation is as follows:

[0076] W(St)=A·sim(S,title)+B·||v s ||

[0077] Among them, sim(S, title) represents the cosine similarity between sentence S and title title, v s It is represented as a vector representation of sentence S, where A and B are weight coefficients;

[0078] In step 2-2, sentences at different positions have different importance and weights. Different weights are assigned to the sentence vectors at different positions, and the sentence position features are output. The example formula for the corresponding operation is as follows:

[0079]

[0080] Where n is the number of paragraphs in the text, x is the number of sentences in the first paragraph, y is the number of sentences in the last paragraph, S is the number of sentences in the text, i and j are the positions of the sentences in the first and last paragraphs, e1, e2, β1, and β2 are weight coefficients, and p represents the parameters of different sentence positions.

[0081] Step 2-3, use the bidirectional multi-layer GRU neural network to obtain the final semantic relationship of the text, input the sentence vector into the bidirectional GRU model, and convert it into an ordered sequence x i =(x i,1 ,x i,2 ,...,x i,d ), the sequence length is d, and for time step t, the hidden state is updated. The example formula is as follows:

[0082]

[0083] in, is the forward hidden state, Backward hidden state.

[0084] Then its final hidden state is expressed as: The last hidden state of the bidirectional GRU is processed to obtain the final semantic features of the sentence. The example formula is as follows:

[0085]

[0086] in, and denote the last layer output of the forward and reverse hidden state vectors, respectively.

[0087] Step 3: Adjust the feature dimensions of Steps 1 and 2, fuse the sentence features with the similarity matrix using a weighted fusion method, iteratively optimize the weight coefficients in the sentence feature weight calculation, and obtain the sentence feature weight calculation formula with the optimal weight coefficient. The specific process of sentence feature fusion includes:

[0088] In step 3-1, principal component analysis and linear layer are used to reduce the dimension and project it.

[0089] Step 3-2: Calculate the similarity between sentences. The example formula is as follows:

[0090]

[0091] Among them, S i and S j They represent sentence i and sentence j respectively, and E(S) represents the sentence vector.

[0092] Step 3-3: perform feature fusion to obtain sentence feature weights, and normalize the features using the comprehensive weight factors. The corresponding operation example formula is as follows:

[0093] W=μW ConSim +c||W(w)||·h i,fin +dW(St)+qW loc (S)+ρh i,fin

[0094] Among them, W ConSim is the similarity matrix between sentences, μ, c, d, q, ρ are weight coefficients (the weight coefficients are calculated through continuous iteration to obtain the optimal weight coefficients).

[0095] Step 4: Calculate the relevance and redundancy of the selected key sentences using the MMR algorithm, and output the MMR value of the candidate key sentence. The summary is then added, and the sentence set is updated. The calculation is iterated until the key sentence length or number requirements are met. The MMR algorithm removes redundancy, and the specific process includes:

[0096] In step 4-1, the sentences are sorted in descending order according to the sentence feature weights, and some sentences are selected as candidate key sentences according to the ratio.

[0097] In step 4-2, the MMR algorithm is used to calculate the relevance and redundancy of the candidate key sentences to obtain the MMR value of the candidate key sentences. The corresponding operation example formula is as follows:

[0098]

[0099] Among them, MMR(s) is the MMR value of the candidate key sentence s, D is the target summary, is a set of candidate key sentences, Sim(s,D) represents the similarity between the candidate key sentence s and the target summary D, Sim(s,s') represents the maximum similarity between the candidate key sentence s and all other candidate key sentences s', and η is the weight coefficient.

[0100] In step 4-3, the candidate key sentences with the highest MMR value are added to the key sentence set, and the iterative calculation is stopped until the key sentence length or number requirements are met to obtain the key sentence.

[0101] In step 5, the text data includes several text units. The key sentences obtained in step 4 are used to obtain the contextual embedding representation of their words using the Word2Vec model. Hierarchical clustering is then performed on the text units to obtain unregistered words. These words are marked as incorrect words, and the contextual embedding of potential replacement words is generated for them. These words are replaced accordingly until the similarity of the replacement words exceeds a threshold, resulting in the replaced text. The dynamic error correction mechanism specifically includes the following steps:

[0102] In step 5-1, the key sentence obtained in step 4 is used to generate contextual embedding representations for the words in the key sentence using the Word2Vec model to obtain the distance between each text unit. The example formula for the corresponding operation is as follows:

[0103]

[0104] Among them, e i and e j are the embedding vectors of the i-th and j-th text units respectively.

[0105] In step 5-2, hierarchical clustering is performed on the text units based on the distance between them, and the similarity between clusters is defined. A dynamic threshold ε is then set based on the maximum and minimum similarities within the clusters. The example formula for the corresponding operation is as follows:

[0106]

[0107] Among them, C k and C lThere are two clusters, |C| represents the number of elements in the cluster, is the weight coefficient.

[0108] In step 5-3, the similarity of each cluster center is calculated to identify potential unregistered words and to mislabel them. The example formula for the corresponding operation is as follows:

[0109]

[0110] in, is the cluster center, is the similarity threshold.

[0111] Step 5-4: For words marked as errors, generate contextual embeddings of their potential alternative words e alt,i , where i = 1, 2, ..., M. If the similarity of the replacement word is higher than the set threshold, that is, S(C k ,e alt,j )>ε, then C k =C k ∪{e alt,j}.

[0112] Get replacement text

[0113] Step 6: Measure the replacement text in step 5 The replacement loss of the key sentence in step 4. Train the improved PEGASUS model (pegasus-base-chinese-cluecorpussmall). The training data includes the processed replacement texts with different replacement words and the reference summary. The replacement text is input into the PEGASUS model for encoding and decoding. The beam search strategy is used to output the generated summary under supervision and unsupervised conditions. The similarity between the reference summary and the generated summary under supervision and the similarity between the summaries of different replacement instances under unsupervised conditions are recorded as a whole as the contrast loss. The supervised and unsupervised cases are contrasted and learned, and the model is iteratively optimized to obtain the trained improved PEGASUS model. The summary with the smallest loss is decoded and output. The specific process of contrastive learning optimization includes:

[0114] Step 6-1, replace the text Compare this with the key sentences E extracted from the original text and calculate the overall replacement loss to guide the model's error correction strategy. The example formula for the corresponding operation is as follows:

[0115]

[0116] Among them, e i For the original word, As an alternative word, Loss function, m is the number of alternative words.

[0117] To cope with the dynamic impact of the error correction strategy, a cross-entropy loss is used to consider multiple alternative words. The example formula for the corresponding operation is as follows:

[0118]

[0119] In step 6-2, the replaced text in the training data is input into the improved PEGASUS model for encoding.

[0120] In step 6-3, beam search is used to perform decoding in both supervised and unsupervised situations. The beam width B is set. At each time step, the probability distribution of the next word is expanded from all current candidate sequences to obtain the total score of each expanded sequence. The example formula for the corresponding operation is as follows:

[0121]

[0122] Among them, w t is the word generated at time step t, and n is the total length of the sequence.

[0123] In step 6-4, perform summary generation under supervision and calculate the cosine similarity between the generated summary and the reference summary to evaluate semantic similarity. The example formula for the corresponding operation is as follows:

[0124] L sc (g,r)=1-cos(f g ,f r )

[0125] Among them, f g and f r Represent the embedding representations of the generated summary and the reference summary respectively.

[0126] To further enhance the accuracy of contrastive learning, we analyze the syntax tree using the structural feature function S() to evaluate structural similarity. The example formula for the corresponding operation is as follows:

[0127] L st (g,r)=||S(f g )-S(f r )||

[0128] In step 6-5, perform summary generation in an unsupervised manner, compare the similarities between the generated summaries of different replacement instances, and maintain the consistency of the generated summaries. The example formula for the corresponding operation is as follows:

[0129]

[0130] Among them, g jrepresents the unsupervised generated instance of the j-th replacement word, and M represents the number of generated instances.

[0131] Update the loss function and combine the loss functions of the two cases. The example formula of the corresponding operation is as follows:

[0132] Loss2(g,r)=tL sc (g,r)+kL st (g,r)+γL con (g)

[0133] Among them, t, k, and γ are weight coefficients;

[0134] To determine whether the comprehensive loss is the best, we need to continuously adjust the parameters and iterate until the comprehensive loss is minimized. This means that the generated summary is more fluent, comprehensive, and suitable for practical applications. The example formula for the corresponding operation is as follows:

[0135]

[0136] Among them, θ is the weight coefficient.

[0137] Step 7: Input the replacement text into the trained improved PEGASUS model and output the summary.

[0138] An embodiment of the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded into the processor, the method for generating a two-stage text summary based on the PEGASUS model and dynamic error correction is implemented.

[0139] An embodiment of the present invention also provides a storage medium storing a computer program. When the computer program is executed by a processor, it implements any one of the two-stage text summary generation methods based on the PEGASUS model and dynamic error correction.

Claims

1. A two-stage text summarization method based on the PEGASUS model and dynamic error correction, characterized in that: The following steps are involved: Step 1: Obtain the text data to be processed and preprocess it to obtain the title, sentence vector, word vector, and title word vector; Step 2: Introduce the title word vector and perform weighted average calculation of the word vector weight. Select keywords based on the word vector weight to obtain keyword features. Calculate the similarity between the sentence vector and the title to obtain the title features; Assign different weights to sentence vectors at different positions to obtain sentence position features; Process the sentence vector through a neural network and output the final semantic features; Step 3: Calculate the similarity between sentences to obtain a sentence similarity matrix; fuse the sentence similarity matrix, keyword features, title features, sentence position features, and final semantic features to obtain sentence feature weights; Step 4: Select sentences based on sentence feature weights to obtain candidate key sentences; calculate the MMR value of the candidate key sentences using the MMR algorithm, add the candidate key sentences with the highest MMR value to the key sentence set, and iterate the calculation until the sentence set meets the key sentence length or number requirements, and then obtain the key sentences; Step 5: The text data consists of several text units. The key sentences are embedded in the context using the Word2Vec model. The distance between each text unit is calculated, and the text units are hierarchically clustered based on the distance between the text units. Step 6: Calculate the similarity of each cluster center to identify potential unregistered words, mark the unregistered words as incorrect words, generate contextual embedding representations of potential alternative words for the incorrectly marked words, and calculate the similarity between the alternative words and the incorrectly marked words based on the contextual embedding representations of the alternative words. When the similarity is higher than the threshold, replace the incorrectly marked words with the alternative words to obtain the replacement text, calculate the replacement loss of the replacement text, and correct the replacement words using the replacement loss. Step 7: Input the replacement text into the trained PEGASUS model and output the summary; The PEGASUS model training process is as follows: The training data includes the replacement texts of different replacement words processed from the text data and the corresponding reference summaries of the text data. The replacement texts in the training data are input into the PEGASUS model, and the generated summaries are output using a beam search strategy in both supervised and unsupervised conditions. The similarity between the reference summary and the generated summary in the supervised condition and the similarity between the generated summaries after different replacement words in the unsupervised condition are calculated, and the overall similarity is recorded as the contrastive loss. The comprehensive loss is calculated by replacing the loss and comparing the loss, and the weight coefficients, dynamic thresholds and similarity thresholds are adjusted until the comprehensive loss reaches the minimum or the number of iterations reaches the maximum, thus obtaining a trained PEGASUS model.

2. The two-stage text summarization method according to claim 1, characterized in that: The preprocessing in step 1 includes: removing stop words, segmenting words, and marking parts of speech on the acquired text data to obtain a text dataset; using the text dataset as input, using the BERT model to generate sentence vectors; and using the Word2Vec model to train external data to generate word vectors.

3. The two-stage text summarization method according to claim 2, characterized in that: In step 2, the weight calculation formula of the word vector is: W(w)=αTF_IDF(w,d)+β||v w ||+γTR(w)+δI T (w) Among them, W(w) represents the weight of the word vector, ||v w ||Word vector v representing word w w The modulus length, TR(w) indicates the importance of word w calculated by the TextRank algorithm, I T (w) represents the judgment parameter of whether word w belongs to the title word vector, α, β, λ, δ are weight coefficients; TF_IDF(w,d) represents the frequency and distribution of word w in the text data, and the calculation formula is: Where TF_IDF(w,d) represents the frequency and distribution of word w in the text data, tf(w,d) represents the frequency of word w in sentence S in document d, df(w) represents the number of documents in which word w appears in all documents, and N is the total number of all documents. The calculation formula for the title feature W(St) obtained based on the similarity between the sentence vector and the title is: W(St)=A·sim(S,title)+B·||v s || Among them, sim(S, title) represents the cosine similarity between sentence S and title title, v s It is represented as a vector representation of sentence S, where A and B are weight coefficients; The calculation formula of sentence position feature is: Among them, n is the number of paragraphs in the article, x is the number of sentences in the first paragraph, y is the number of sentences in the last paragraph, i and j are the positions of the sentences in the first and last paragraphs, e1, e2, β1, and β2 are weight coefficients, and p represents the parameters of different sentence positions.

4. The two-stage text summarization method according to claim 2, characterized in that: In step 2, the sentence vector is processed by a bidirectional multi-layer GRU neural network to output the final semantic features. Specifically, the sentence vector is input into the bidirectional multi-layer GRU neural network and converted into an ordered sequence x i =(x i,1 ,x i,2 ,...,x i,d ), the sequence length is d, the last hidden state of the bidirectional multi-layer GRU neural network is processed to obtain the final semantic feature h of the sentence i,fin : in, and Represents the last layer output of the forward and reverse hidden state vectors.

5. The two-stage text summarization method according to claim 4, characterized in that: The calculation formula of sentence feature weight W in step 3 is: W=μW ConSim +c||W(w)||·h i,fin +dW(St)+qW loc (S)+ρh i,fin Among them, W ConSim is the similarity between sentences, μ, c, d, q, ρ are weight coefficients.

6. The two-stage text summarization method according to claim 5, characterized in that: In step 4, the sentences are sorted in descending order according to the sentence feature weights, and sentences are selected as candidate key sentences according to the ratio. The MMR algorithm is used to calculate the relevance and redundancy of the candidate key sentences to obtain the MMR value of the candidate key sentences. The calculation formula is: Among them, MMR(s) is the MMR value of the candidate key sentence s, D is the target summary, is a set of candidate key sentences, Sim(s,D) represents the similarity between the candidate key sentence s and the target summary D, Sim(s,s') represents the maximum similarity between the candidate key sentence s and all other candidate key sentences s', and η is the weight coefficient.

7. The two-stage text summarization method according to claim 6, characterized in that: In step 6, the replacement loss between the replacement text and the key sentences of the original text is calculated using the following formula: Among them, E is the key sentence of the original text, To replace the text, e i is the i-th original word in the key sentence of the original text, is the replacement word for the i-th original word, Loss function, m is the number of alternative words.

8. The two-stage text summarization method according to claim 7, characterized in that: The similarity between the reference summary and the generated summary under supervision in step 7 includes semantic similarity and structural similarity; Semantic similarity is expressed by cosine similarity, and the calculation formula is: L sc (g,r)=1-cos(f g ,f r ) Among them, g represents the generated summary, r represents the reference summary; f g represents the embedding representation of the generated summary, f r Embedding representation of the reference summary; The formula for calculating structural similarity is: L st (g,r)=||S(f g )-S(f r )|| Among them, S(f g ) represents the structural feature function of the embedding representation of the generated summary, S(f r ) represents the structural feature function of the embedding representation of the reference summary; The similarity calculation formula between the summaries generated after different replacement words in an unsupervised situation is: Among them, g j represents the unsupervised generation instance of the j-th replacement word, and M represents the number of generated instances; The contrast loss is: Loss2(g,r)=tL sc (g,r)+kL st (g,r)+γL con (g) Among them, t, k, and γ are weight coefficients; The comprehensive loss is: Among them, θ is the weight coefficient.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images