Two-stage text abstract generation method based on PEGASUS model and dynamic error correction

Abstract

The invention discloses a two-stage text abstract generation method based on a PEGASUS model and dynamic error correction, and the method comprises the steps: obtaining to-be-processed text data, carrying out the preprocessing, obtaining all features, and carrying out the fusion of all features, and obtaining a sentence weight; selecting sentences according to the sentence weights, and selecting key sentences of the abstract; performing hierarchical clustering on the key sentences; identifying potential unlisted words, marking the unlisted words as wrong words, replacing the words marked as wrong words with alternative words, and performing error correction on the alternative words through replacement loss; calculating the comparison loss of the replacement text through a PEGASUS model; and calculating the comprehensive loss through the replacement loss and the comparison loss, adjusting each weight coefficient, the dynamic threshold value and the similarity threshold value until the comprehensive loss reaches the minimum or the number of iterations reaches the maximum, and outputting the abstract. The unregistered words are replaced by different alternative words through a'dynamic error correction mechanism ', so that the optimal selection of the abstract is realized, and the accuracy of generating the abstract is improved.

Description

Technical Field

[0001] The present invention relates to text summary generation, and in particular to a two-stage text summary generation method based on a PEGASUS model and dynamic error correction. Background Art

[0002] The two-stage text summarization task mainly involves the following four aspects of technology: First, key sentence extraction is one of the basic tasks of two-stage summary generation, which aims to extract important sentences with dense information from the original text. Commonly used technologies include graph-based ranking algorithms such as TextRank, deep learning methods such as BERT embedding similarity comparison, and machine learning methods such as SVM. These methods focus on identifying and extracting sentences most relevant to the topic and perform well in information extraction. Second, text encoding and understanding also play an important role in the two-stage task, especially in deep semantic understanding. Models such as BERT and Transformer are widely used in this stage, which can effectively capture context-related information, thereby improving the model's ability to understand text intent and semantics. Third, abstract summary generation is the second important step in the two-stage process. By using generative pre-trained models such as PEGASUS or T5, the model can semantically reorganize and simplify the extracted key sentences to generate a fluent and concise summary. This stage emphasizes generating natural language text while retaining core information. Fourth, model optimization and integration are important guarantees for ensuring the quality of summary generation. By using techniques such as knowledge distillation, adversarial training, and multi-task learning, the generalization performance and generation accuracy of the model can be improved.

[0003] The main problems in current tasks are as follows: difficulty in sentence selection due to the diversity of text topics; the balance between the naturalness and accuracy of generated summaries; the existence of unregistered words and exposure bias; and computational efficiency issues when processing long texts. Summary of the invention

[0004] Purpose of the invention: In view of the above shortcomings, the present invention provides a two-stage text summary generation method based on the PEGASUS model and dynamic error correction with high accuracy.

[0005] Technical solution: To solve the above problems, the present invention adopts a two-stage text summary generation method based on the PEGASUS model and dynamic error correction, comprising the following steps:

[0006] Step 1: Get the text data to be processed and preprocess it to obtain the title, sentence vector, word vector, and title word vector;

[0007] Step 2: Introduce the title word vector to calculate the weight of the word vector by weighted average, select keywords according to the weight of the word vector, and obtain keyword features; calculate the similarity between the sentence vector and the title to obtain the title feature; assign different weights to the sentence vectors at different positions to obtain the sentence position feature; process the sentence vector through a neural network and output the final semantic feature;

[0008] Step 3: Calculate the similarity between sentences to obtain a sentence similarity matrix; fuse the sentence similarity matrix, keyword features, title features, sentence position features, and final semantic features to obtain sentence feature weights;

[0009] Step 4: Select sentences according to sentence feature weights to obtain candidate key sentences; calculate the MMR value of the candidate key sentences through the MMR algorithm, add the candidate key sentences with the highest MMR value to the key sentence set, and iterate the calculation until the sentence set meets the key sentence length or number requirements to obtain the key sentences;

[0010] Step 5: The text data includes several text units. The key sentences are embedded in the context through the Word2Vec model, the distance between each text unit is calculated, and the text units are hierarchically clustered according to the distance between the text units.

[0011] Step 6: Calculate the similarity of each cluster center to identify potential unregistered words, mark the unregistered words as wrong words, generate context embedding representations of potential substitute words for the wrongly marked words, and calculate the similarity between the substitute words and the wrongly marked words based on the context embedding representations of the substitute words. When the similarity is higher than the threshold, replace the wrongly marked words with the substitute words to obtain the replacement text, calculate the replacement loss of the replacement text, and correct the substitute words through the replacement loss.

[0012] Step 7: Input the replacement text into the trained PEGASUS model and output the summary;

[0013] The training process of the PEGASUS model is as follows: obtain training data, which includes replacement texts of different replacement words processed from text data and reference summaries corresponding to the text data, input the replacement texts in the training data into the PEGASUS model, and use the beam search strategy to output the generated summary in supervised and unsupervised conditions, calculate the similarity between the reference summary and the generated summary in the supervised condition and the similarity between the generated summaries after different replacement words in the unsupervised condition, and record the whole as the contrast loss;

[0014] The comprehensive loss is calculated by replacing the loss and comparing the loss, and the weight coefficients, dynamic thresholds and similarity thresholds are adjusted until the comprehensive loss reaches the minimum or the number of iterations reaches the maximum, thus obtaining a trained PEGASUS model.

[0015] Furthermore, the preprocessing in step 1 includes: removing stop words, segmenting words, and marking parts of speech on the acquired text data to obtain a text data set; using the text data set as input and using the sentence vectors generated by the BERT model; and using the Word2Vec model to train external data to generate word vectors.

[0016] Furthermore, in step 2, the weight calculation formula of the word vector is:

[0017] W(w)=αTF_IDF(w,d)+β||v w ||+γTR(w)+δI T (w)

[0018] Among them, W(w) represents the keyword feature weight, ||v w ||Word vector v representing word w w The modulus length, TR(w) indicates the importance of word w calculated by the TextRank algorithm, I T (w) indicates whether word w belongs to the title word vector, α, β, λ, δ are weight coefficients; TF_IDF(w,d) indicates the frequency and distribution of word w in text data, and the calculation formula is:

[0019]

[0020] Among them, TF_IDF(w,d) represents the frequency and distribution of word w in text data, tf(w,d) represents the frequency of word w in sentence S in document d, df(w) represents the number of documents in which word w appears in all documents, and N is the total number of all documents;

[0021] The calculation formula of the title feature W(St) is obtained based on the similarity between the sentence vector and the title:

[0022] W(St)=A·sim(S,title)+B·||v s ||

[0023] Among them, sim(S, title) represents the cosine similarity between sentence S and title title, v s It is represented as a vector representation of sentence S, where A and B are weight coefficients;

[0024] The calculation formula of sentence position feature is:

[0025]

[0026] Among them, n is the number of paragraphs in this article, x is the number of sentences in the first paragraph, y is the number of sentences in the last paragraph, i and j are the positions of the sentences in the first and last paragraphs, e1, e2, β1, β2 are weight coefficients, and p represents the parameters of different sentence positions.

[0027] Furthermore, in step 2, the sentence vector is processed by a bidirectional multi-layer GRU neural network to output the final semantic features, specifically: the sentence vector is input into the bidirectional multi-layer GRU neural network and converted into an ordered sequence x i =(x i,1 ,x i,2 ,...,x i,d ), the sequence length is d, and the last hidden state of the bidirectional multi-layer GRU neural network is processed to obtain the final semantic feature h of the sentence i,fin :

[0028]

[0029] in, and Represents the last layer output of the forward and reverse hidden state vectors.

[0030] Furthermore, the calculation formula of the sentence feature weight W in step 3 is:

[0031] W=μW ConSim +c||W(w)||·h i,fin +dW(St)+qW loc (S)+ρh i,fin

[0032] Among them, W ConSim is the similarity matrix between sentences, μ, c, d, q, ρ are weight coefficients.

[0033] Furthermore, in step 4, the sentences are sorted in descending order according to the sentence feature weights, and the sentences are selected as candidate key sentences according to the ratio. The MMR algorithm is used to calculate the relevance and redundancy of the candidate key sentences to obtain the MMR value of the candidate key sentences. The calculation formula is:

[0034]

[0035] Among them, MMR(s) is the MMR value of the candidate key sentence s, D is the target summary, is a set of candidate key sentences, Sim(s,D) represents the similarity between the candidate key sentence s and the target summary D, Sim(s,s') represents the maximum similarity between the candidate key sentence s and all other candidate key sentences s', and η is the weight coefficient.

[0036] Furthermore, in step 6, the replacement loss between the replacement text and the key sentences of the original text is calculated, and the calculation formula is:

[0037]

[0038] Among them, E is the key sentence of the original text, To replace the text, e i is the i-th original word in the key sentence of the original text, is the replacement word for the i-th original word, Loss function, m is the number of alternative words.

[0039] Furthermore, the similarity between the reference summary and the generated summary in the supervised case in step 7 includes semantic similarity and structural similarity;

[0040] Semantic similarity is expressed by cosine similarity, and the calculation formula is:

[0041] L sc (g,r)=1-cos(f g ,f r )

[0042] Among them, g represents the generated summary, r represents the reference summary; f g represents the embedding representation of the generated summary, f r Embedding representation representing the reference summary;

[0043] The formula for calculating structural similarity is:

[0044] L st (g,r)=||S(f g )-S(f r )||

[0045] Among them, S(f g ) represents the structural feature function of the embedding representation of the generated summary, S(f r ) represents the structural feature function of the embedding representation of the reference summary;

[0046] The similarity calculation formula between abstracts after replacing words in an unsupervised situation is:

[0047]

[0048] Among them, g j represents the unsupervised generated instance of the j-th replacement word, and M represents the number of generated instances;

[0049] The contrast loss is:

[0050] Loss2(g,r)=tL sc (g,r)+kL st(g,r)+γL con (g)

[0051] Among them, t, k, and γ are weight coefficients;

[0052] The comprehensive loss is:

[0053]

[0054] Among them, θ is the weight coefficient.

[0055] The present invention also adopts a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0056] The present invention also adopts a computer-readable storage medium on which a computer program is stored, and the computer program implements the steps of the above method when executed by a processor.

[0057] Beneficial effects: Compared with the prior art, the significant advantages of the present invention are that a dynamic error correction mechanism module is constructed, hierarchical clustering is dynamically applied, and different substitute words are effectively replaced for unregistered words to achieve the optimal selection of the summary; the present invention uses both unsupervised and supervised methods to generate summaries, and constructs a comparative learning framework to ensure the accuracy and naturalness of the output summary; the present invention uses replacement loss and contrast loss as comprehensive losses to train the model, and can obtain summary texts with accurate expression, fluent sentences, and in line with human reading habits; the present invention uses a bidirectional multi-layer GRU neural network to extract features from key sentences and integrate them into semantic features, and the model has fewer parameters, high efficiency and good effect; the present invention divides text summary generation into two stages, and uses the extraction stage to extract and integrate key sentences, optimizes the text structure, reduces the relative input of the abstract stage, and improves the abstract efficiency and the accuracy of summary generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a flowchart of the abstract generation method of the present invention.

[0059] Figure 2 It is the first stage model framework diagram in the present invention.

[0060] Figure 3 It is the framework diagram of the second stage model in the present invention. DETAILED DESCRIPTION

[0061] like Figure 1 and Figure 2 As shown, in this embodiment, a two-stage text summary generation method based on the PEGASUS model and dynamic error correction includes the following steps:

[0062] Step 1: Obtain the text data to be processed, perform preprocessing on the obtained text data such as removing stop words, segmenting, and marking parts of speech to obtain text data sets and titles, use the BERT model to process the text data sets to generate sentence vectors, and use the Word2Vec model to train the text data sets and titles with external data to generate word vectors and title word vectors. Use the TextRank algorithm to construct a weighted undirected graph to obtain word vector weights, and introduce the title word vector for weighted averaging to output keyword features. The specific process of keyword feature acquisition includes:

[0063] Step 1-1, preprocess the acquired text data, remove stop words, segment words, mark parts of speech, and divide the data set into training set, verification set, and test set;

[0064] Step 1-2, use the sentence vector generated by the BERT model to perform operations and retain the overall information of the sentence. i The sentence vector E(S) is obtained through the BERT model i ). The Word2Vec model is used to generate word vectors and title vectors by introducing a large amount of external data to enrich the semantic features of the text.

[0065] Step 1-3, use the TextRank algorithm to construct a weighted undirected graph to obtain the word vector weight, and introduce the title word vector for weighted averaging. For each word w in sentence S, the frequency of occurrence in document d is tf(w,d), and the number of documents in which word w appears in document set D is df(w), and document set D includes all documents. The frequency and distribution of word w, TF_IDF, is as follows:

[0066]

[0067] Where N is the total number of documents.

[0068] The title word vector is introduced to optimize the calculation of word vector weights. The example formula of the corresponding operation is as follows:

[0069]

[0070] Calculate the word vector weights. The example formula for the corresponding operation is as follows:

[0071] W(w)=αTF_IDF(w,d)+βv w ||+γTR(w)+δI T (w)

[0072] Among them, W(w) represents the word vector weight, ||v w ||Word vector v representing word w w The modulus length, TR(w) indicates the importance of word w calculated by the TextRank algorithm, IT (w) represents the judgment parameter of whether word w belongs to the title word vector, α, β, λ, δ are weight coefficients;

[0073] According to the word vector weight, the word with the highest weight is selected as the keyword. The word vector weight of the keyword is the keyword feature weight, and the keyword feature weight is used as the keyword feature;

[0074] Step 2: Extract relevant sentence features, measure the relationship between the sentence vector and the title, assign different weights to sentences in different positions, deeply understand the meaning of sentences and obtain the semantic relationship of text. The specific process of extracting sentence-related features includes:

[0075] Step 2-1, measure the similarity between the sentence vector of the text and the title, and output the title features. The example formula of the corresponding operation is as follows:

[0076] W(St)=A·sim(S,title)+B·||v s ||

[0077] Among them, sim(S, title) represents the cosine similarity between sentence S and title title, v s It is represented as a vector representation of sentence S, where A and B are weight coefficients;

[0078] Step 2-2: The importance and weight of sentences at different positions are different. Different weights are assigned to the sentence vectors at different positions, and the sentence position features are output. The example formula for the corresponding operation is as follows:

[0079]

[0080] Among them, n is the number of paragraphs in the text, x is the number of sentences in the first paragraph, y is the number of sentences in the last paragraph, S is the sentence in the text, i and j are the positions of the sentences in the first and last paragraphs, e1, e2, β1, β2 are weight coefficients, and p represents the parameters of different sentence positions.

[0081] Step 2-3, use the bidirectional multi-layer GRU neural network to obtain the final semantic relationship of the text, input the sentence vector into the bidirectional GRU model, and convert it into an ordered sequence x i =(x i,1 ,x i,2 ,...,x i,d ), the sequence length is d, for time step t, the hidden state is updated, and the example formula is as follows:

[0082]

[0083] in, is the forward hidden state, is the backward hidden state.

[0084] Then its final hidden state is expressed as: The last hidden state of the bidirectional GRU is processed to obtain the final semantic features of the sentence. The example formula is as follows:

[0085]

[0086] in, and denote the last layer output of the forward and reverse hidden state vectors respectively.

[0087] Step 3: Adjust the feature dimensions of step 1 and step 2, fuse the sentence features with the similarity matrix using weighted fusion, iteratively optimize the weight coefficients in the sentence feature weight calculation, and obtain the sentence feature weight calculation formula with the optimal weight coefficient. The specific process of sentence feature fusion includes:

[0088] Step 3-1, use principal component analysis and linear layer to reduce the dimension and project it.

[0089] Step 3-2, calculate the similarity between sentences. The example formula is as follows:

[0090]

[0091] Among them, S i and S j They represent sentence i and sentence j respectively, and E(S) represents the sentence vector.

[0092] Step 3-3, perform feature fusion, obtain sentence feature weights, and normalize the features using the comprehensive weight factors. The corresponding operation example formula is as follows:

[0093] W=μW ConSim +c||W(w)||·h i,fin +dW(St)+qW loc (S)+ρh i,fin

[0094] Among them, W ConSim is the similarity matrix between sentences, μ, c, d, q, ρ are weight coefficients (the weight coefficients are calculated through continuous iteration to obtain the optimal weight coefficients).

[0095] Step 4: Calculate the relevance and redundancy of the selected key sentences according to the MMR algorithm, and output the MMR value of the candidate key sentence. Add the summary, update the sentence set, and iterate until the key sentence length or number requirements are met. The MMR algorithm removes redundancy, and the specific process includes:

[0096] Step 4-1, sort the sentences in descending order according to the sentence feature weights, and select some sentences as candidate key sentences according to the ratio.

[0097] Step 4-2, using the MMR algorithm, calculate the relevance and redundancy of the candidate key sentences to obtain the MMR value of the candidate key sentences. The corresponding operation example formula is as follows:

[0098]

[0099] Among them, MMR(s) is the MMR value of the candidate key sentence s, D is the target summary, is a set of candidate key sentences, Sim(s,D) represents the similarity between the candidate key sentence s and the target summary D, Sim(s,s') represents the maximum similarity between the candidate key sentence s and all other candidate key sentences s', and η is the weight coefficient.

[0100] Step 4-3, add the candidate key sentence with the highest MMR value to the key sentence set, iterate the calculation until the key sentence length or number requirement is met, and then obtain the key sentence.

[0101] Step 5: The text data includes several text units. The key sentences obtained in step 4 are used to obtain the contextual embedding representation of their words using the Word2Vec model, and the text units are hierarchically clustered to obtain unregistered words. They are marked as incorrect words, and the contextual embedding of potential alternative words is generated for them. They are replaced accordingly until the similarity of the alternative words exceeds the threshold, and the replaced text is obtained. The dynamic error correction mechanism includes the following specific processes:

[0102] Step 5-1: Use the Word2Vec model to generate context embedding representations for the words in the key sentences obtained in step 4, and obtain the distance between each text unit. The example formula for the corresponding operation is as follows:

[0103]

[0104] Among them, e i and e j are the embedding vectors of the i-th and j-th text units respectively.

[0105] Step 5-2, hierarchically cluster the text units according to the distance between the text units, and define the similarity between the clusters. Then set the dynamic threshold ε according to the maximum similarity and the minimum similarity within the cluster. The example formula of the corresponding operation is as follows:

[0106]

[0107] Among them, C k and C lare two clusters, |C| represents the number of elements in the cluster, is the weight coefficient.

[0108] Step 5-3, calculate the similarity of each cluster center to identify potential unknown words and mislabel the unknown words. The example formula of the corresponding operation is as follows:

[0109]

[0110] in, is the cluster center, is the similarity threshold.

[0111] Step 5-4: Generate context embeddings of potential alternative words for words marked as errors alt,i , where i = 1, 2, ..., M. If the similarity of the replacement word is higher than the set threshold, that is, S(C k ,e alt,j )>ε, then C k =C k ∪{e alt,j}.

[0112] Get replacement text

[0113] Step 6: Evaluate the replacement text in step 5 Replacement loss with the key sentence in step 4. Train the improved PEGASUS model (pegasus-base-chinese-cluecorpussmall). The training data includes the processed replacement texts with different replacement words and the reference summary. The replacement text is input into the PEGASUS model for encoding and decoding. The generated summary is output using the beam search strategy under supervision and unsupervised conditions. The similarity between the reference summary and the generated summary under supervision and the similarity between the summaries of different replacement instances under unsupervised conditions are recorded as a whole as the contrast loss. Perform contrastive learning on the supervised and unsupervised conditions, iteratively optimize the model, obtain the trained improved PEGASUS model, and decode and output the summary with the smallest loss. Contrastive learning optimization, the specific process includes:

[0114] Step 6-1, replace the text Compare with the key sentences E extracted from the original text and calculate the overall replacement loss to guide the model's error correction strategy. The example formula for the corresponding operation is as follows:

[0115]

[0116] Among them, e i For the original word, As an alternative word, Loss function, m is the number of alternative words.

[0117] To cope with the dynamic impact of the error correction strategy, a cross entropy loss is used to consider multiple alternative words. The example formula for the corresponding operation is as follows:

[0118]

[0119] Step 6-2, input the replaced text in the training data into the improved PEGASUS model for encoding.

[0120] Step 6-3, use beam search to perform supervised and unsupervised decoding, set the beam width B, and in each time step, expand the probability distribution of the next word from all current candidate sequences to obtain the total score of each sequence after expansion. The example formula of the corresponding operation is as follows:

[0121]

[0122] Among them, w t is the word generated at time step t, and n is the total length of the sequence.

[0123] Step 6-4, perform summary generation under supervision, and calculate the cosine similarity between the generated summary and the reference summary to evaluate the semantic similarity. The example formula for the corresponding operation is as follows:

[0124] L sc (g,r)=1-cos(f g ,f r )

[0125] Among them, f g and f r Represent the embedding representations of the generated summary and the reference summary, respectively.

[0126] To further enhance the accuracy of contrastive learning, the syntax tree is analyzed through the structural feature function S() to evaluate the structural similarity. The example formula for the corresponding operation is as follows:

[0127] L st (g,r)=||S(f g )-S(f r )||

[0128] Step 6-5, generate summaries in an unsupervised manner, compare the similarities between the generated summaries of different replacement instances, and maintain the consistency of the generated summaries. The example formula of the corresponding operation is as follows:

[0129]

[0130] Among them, g jrepresents the unsupervised generated instance of the j-th replacement word, and M represents the number of generated instances.

[0131] Update the loss function and merge the loss functions of the two cases. The example formula of the corresponding operation is as follows:

[0132] Loss2(g,r)=tL sc (g,r)+kL st (g,r)+γL con (g)

[0133] Among them, t, k, and γ are weight coefficients;

[0134] To judge whether the comprehensive loss is the best, we need to continuously adjust the parameters and iterate until the comprehensive loss is the minimum, which means that the generated summary is more fluent, more comprehensive, and more suitable for practical applications. The example formula for the corresponding operation is as follows:

[0135]

[0136] Among them, θ is the weight coefficient.

[0137] Step 7: Input the replacement text into the trained improved PEGASUS model and output the summary.

[0138] An embodiment of the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is loaded into the processor, any one of the two-stage text summary generation methods based on the PEGASUS model and dynamic error correction is implemented.

[0139] An embodiment of the present invention also provides a storage medium storing a computer program. When the computer program is executed by a processor, it implements any one of the two-stage text summary generation methods based on the PEGASUS model and dynamic error correction.

Claims

1. A two-stage text summary generation method based on PEGASUS model and dynamic error correction, characterized in that: The following steps are involved: Step 1: Get the text data to be processed and preprocess it to obtain the title, sentence vector, word vector, and title word vector; Step 2: Introduce the title word vector to calculate the weight of the word vector by weighted average, select keywords according to the weight of the word vector, and obtain keyword features; Calculate the similarity between the sentence vector and the title to obtain the title features; Assign different weights to sentence vectors at different positions to obtain sentence position features; Process the sentence vector through a neural network and output the final semantic features; Step 3: Calculate the similarity between sentences to obtain a sentence similarity matrix; fuse the sentence similarity matrix, keyword features, title features, sentence position features, and final semantic features to obtain sentence feature weights; Step 4: Select sentences according to sentence feature weights to obtain candidate key sentences; calculate the MMR value of the candidate key sentences through the MMR algorithm, add the candidate key sentences with the highest MMR value to the key sentence set, and iterate the calculation until the sentence set meets the key sentence length or number requirements to obtain the key sentences; Step 5: The text data includes several text units. The key sentences are embedded in the context through the Word2Vec model, the distance between each text unit is calculated, and the text units are hierarchically clustered according to the distance between the text units. Step 6: Calculate the similarity of each cluster center to identify potential unregistered words, mark the unregistered words as wrong words, generate context embedding representations of potential substitute words for the wrongly marked words, and calculate the similarity between the substitute words and the wrongly marked words based on the context embedding representations of the substitute words. When the similarity is higher than the threshold, replace the wrongly marked words with the substitute words to obtain the replacement text, calculate the replacement loss of the replacement text, and correct the substitute words through the replacement loss. Step 7: Input the replacement text into the trained PEGASUS model and output the summary; The training process of the PEGASUS model is as follows: obtain training data, which includes replacement texts of different replacement words processed from text data and reference summaries corresponding to the text data, input the replacement texts in the training data into the PEGASUS model, and use the beam search strategy to output the generated summary in supervised and unsupervised conditions, calculate the similarity between the reference summary and the generated summary in the supervised condition and the similarity between the generated summaries after different replacement words in the unsupervised condition, and record the whole as the contrast loss; The comprehensive loss is calculated by replacing the loss and comparing the loss, and the weight coefficients, dynamic thresholds and similarity thresholds are adjusted until the comprehensive loss reaches the minimum or the number of iterations reaches the maximum, thus obtaining a trained PEGASUS model.

2. The two-stage text summarization generation method according to claim 1 is characterized in that: The preprocessing in step 1 includes: removing stop words, segmenting words, and marking parts of speech on the acquired text data to obtain a text data set; using the text data set as input and using the sentence vectors generated by the BERT model; and using the Word2Vec model to train external data to generate word vectors.

3. The two-stage text summary generation method according to claim 2 is characterized in that: In step 2, the weight calculation formula of the word vector is: W(w)=αTF_IDF(w,d)+β||v w ||+γTR(w)+δI T (w) Among them, W(w) represents the weight of the word vector, ||v w ||Word vector v representing word w w The modulus length, TR(w) indicates the importance of word w calculated by the TextRank algorithm, I T (w) indicates whether word w belongs to the title word vector, α, β, λ, δ are weight coefficients; TF_IDF(w,d) indicates the frequency and distribution of word w in text data, and the calculation formula is: Among them, TF_IDF(w,d) represents the frequency and distribution of word w in text data, tf(w,d) represents the frequency of word w in sentence S in document d, df(w) represents the number of documents in which word w appears in all documents, and N is the total number of all documents; The calculation formula of the title feature W(St) is obtained based on the similarity between the sentence vector and the title: W(St)=A·sim(S,title)+B·||v s || Among them, sim(S, title) represents the cosine similarity between sentence S and title title, v s It is represented as a vector representation of sentence S, where A and B are weight coefficients; The calculation formula of sentence position feature is: Among them, n is the number of paragraphs in this article, x is the number of sentences in the first paragraph, y is the number of sentences in the last paragraph, i and j are the positions of the sentences in the first and last paragraphs, e1, e2, β1, β2 are weight coefficients, and p represents the parameters of different sentence positions.

4. The two-stage text summarization generation method according to claim 2 is characterized in that: In step 2, the sentence vector is processed by a bidirectional multi-layer GRU neural network to output the final semantic features, specifically: the sentence vector is input into the bidirectional multi-layer GRU neural network and converted into an ordered sequence x i =(x i,1 ,x i,2 ,...,x i,d ), the sequence length is d, the last hidden state of the bidirectional multi-layer GRU neural network is processed to obtain the final semantic feature h of the sentence i,fin : in, and Represents the last layer output of the forward and reverse hidden state vectors.

5. The two-stage text summarization generation method according to claim 4 is characterized in that: The calculation formula of sentence feature weight W in step 3 is: W=μW ConSim +c||W(w)||·h i,fin +dW(St)+qW loc (S)+ρh i,fin Among them, W ConSim is the similarity between sentences, μ, c, d, q, ρ are weight coefficients.

6. The two-stage text summarization generation method according to claim 5, characterized in that: In step 4, the sentences are sorted in descending order according to the sentence feature weights, and the sentences are selected as candidate key sentences according to the ratio. The relevance and redundancy of the candidate key sentences are calculated using the MMR algorithm to obtain the MMR value of the candidate key sentences. The calculation formula is: Among them, MMR(s) is the MMR value of the candidate key sentence s, D is the target summary, is a set of candidate key sentences, Sim(s,D) represents the similarity between the candidate key sentence s and the target summary D, Sim(s,s') represents the maximum similarity between the candidate key sentence s and all other candidate key sentences s', and η is the weight coefficient.

7. The two-stage text summarization generation method according to claim 6, characterized in that: In step 6, the replacement loss between the replacement text and the key sentences of the original text is calculated, and the calculation formula is: Among them, E is the key sentence of the original text, To replace the text, e i is the i-th original word in the key sentence of the original text, is the replacement word for the i-th original word, Loss function, m is the number of alternative words.

8. The two-stage text summarization method according to claim 7, characterized in that: The similarity between the reference summary and the generated summary in the supervised case in step 7 includes semantic similarity and structural similarity; Semantic similarity is expressed by cosine similarity, and the calculation formula is: L sc (g,r)=1-cos(f g ,f r ) Among them, g represents the generated summary, r represents the reference summary; f g represents the embedding representation of the generated summary, f r Embedding representation representing the reference summary; The formula for calculating structural similarity is: L st (g,r)=||S(f g )-S(f r )|| Among them, S(f g ) represents the structural feature function of the embedding representation of the generated summary, S(f r ) represents the structural feature function of the embedding representation of the reference summary; The similarity calculation formula between the summaries generated after different replacement words in an unsupervised situation is: Among them, g j represents the unsupervised generated instance of the j-th replacement word, and M represents the number of generated instances; The contrast loss is: Loss2(g,r)=tL sc (g,r)+kL st (g,r)+γL con (g) Among them, t, k, and γ are weight coefficients; The comprehensive loss is: Among them, θ is the weight coefficient.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images