A Text Summarization Assisted Generation Method Based on Contrastive Learning

Abstract

The present invention discloses a text summary auxiliary generation method based on contrastive learning. First, a candidate summary generation model is constructed, which includes a text embedding module and a source text encoding module; the text embedding module performs vocabulary segmentation, word embedding and position encoding embedding on the source text to obtain the embedding vector of the source text; the source text encoding module is responsible for semantic modeling and feature extraction of the source text to obtain the encoded latent vector. Then, multiple candidate summaries are generated based on the self-attention structure. Then, the encoder network of the candidate summary generation model is used to obtain the sentence vector encoding. Finally, a new sorting loss function is designed based on the contrastive learning idea. Based on the loss function, the semantic similarity of each candidate summary is effectively evaluated according to the source text, and finally the most reasonable text summary of the source text is given. The present invention effectively alleviates the problem of low semantic matching between high-probability candidate summaries and source texts, and can achieve efficient text summary auxiliary generation.

Description

Technical Field

[0001] The present invention belongs to the technical field of deep learning, and in particular relates to a text summary auxiliary generation method. Background Art

[0002] Text summary generation is a relatively subjective task, because each source text actually has many excellent reference summaries. But in fact, most of the existing text summary datasets are single-reference summary datasets, and these reference summaries are not completely correct. Some reference summaries also face problems such as completeness and fluency. The main reason for this is that the reference summaries in the dataset are also manually written, and human thinking is divergent. Different writers have different views on the same article, so the degree of attention to key information in the text is also different. In addition, most text summary generation models based on deep learning use teacher forced training mechanisms and maximum likelihood estimation losses during training, which makes the model tend to generate low-diversity summaries. The exposure bias problem and the lack of multi-reference summary datasets in the training and testing process have aggravated this phenomenon.

[0003] The paper "Liu Y, Liu P. SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization [C] / / Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. 2021: 1065-1072." discloses a text summary generation method based on deep learning. This method constructs a text summary generation framework SimCLS, which formulates the text generation problem as an evaluation problem without references, effectively improving the gap between the learning objectives and evaluation indicators brought about by the currently dominant sequence-to-sequence learning framework. However, this method has the following problems: this method uses a beam search algorithm for decoding generation, and when returning multiple candidate summaries, the candidate summary with the highest degree of match to the source text is hidden behind the high-probability candidate summary, and the candidate summary with the highest degree of semantic match to the source text is often not the candidate summary with the highest matching index. Summary of the invention

[0004] In order to overcome the shortcomings of the prior art, the present invention provides a text summary auxiliary generation method based on contrastive learning. First, a candidate summary generation model capable of obtaining a candidate summary set from a source text is constructed, which includes a text embedding module and a source text encoding module; the text embedding module performs word list segmentation, word embedding and position encoding embedding on the source text to obtain the embedding vector of the source text; the source text encoding module is responsible for semantic modeling and feature extraction of the source text to obtain the encoding latent vector. Then, according to the encoding latent vector, based on the self-attention structure, multiple candidate summaries are generated by setting different sampling parameters of the beam search algorithm. Next, a candidate summary evaluation model is constructed, and the encoder network of the candidate summary generation model is used to encode and embed the candidate summary set, the source text and the reference summary, respectively, to obtain the sentence vector encoding. Further, according to the similarity score index between the candidate summary set and the reference summary, the candidate summaries are evaluated in order from large to small and rearranged. Finally, a new sorting loss function is designed based on the contrastive learning idea. Based on the loss function, each candidate summary is effectively evaluated for semantic similarity according to the source text, and finally the most reasonable text summary of the source text is given. Since the candidate summary evaluation model in the present invention is based on the encoder structure of the generation model, the designed contrastive learning ranking loss function can effectively evaluate the semantic similarity of each candidate summary according to the source text, effectively alleviating the problem of low semantic matching between high-probability candidate summaries and the source text; at the same time, the candidate summary generation model can be effectively extended to the current mainstream generation model based on sequence-to-sequence generation structure, and can realize efficient auxiliary generation of text summaries.

[0005] The technical solution adopted by the present invention to solve the technical problem includes the following steps:

[0006] Step 1: Build a candidate summary generation model;

[0007] The candidate summary generation model constructed consists of two parts, namely, a text embedding module and a source text encoding module. The text embedding module performs vocabulary segmentation, word embedding and position encoding embedding on the source text to obtain the embedding vector of the source text. The source text encoding module is responsible for semantic modeling and feature extraction of the source text.

[0008] Step 2: Based on the constructed candidate summary generation model, a candidate summary set is generated according to the encoding latent vector and the beam search algorithm;

[0009] The decoder is constructed for the encoded latent vector to obtain a set of candidate summaries; the decoder includes a masked self-attention network, a cross-attention network, and a feedforward neural network; the masked self-attention network is used to decode and output the text information of the previous time step and perform forward encoding; the cross-attention network allows the model to decode and generate output from the source text based on the calculation of the encoded latent vector; finally, the output is sent to the feedforward neural network for feature transformation and extraction, thereby generating a set of candidate summaries;

[0010] Step 3: Build a candidate summary evaluation model;

[0011] The candidate summary evaluation model uses the encoder network of the candidate summary generation model to encode and embed the candidate summary set, source text and reference summary respectively to obtain sentence vector encoding; then, according to the similarity score indicator between the candidate summary set and the reference summary, namely the ROUGE-1 score, the candidate summaries are evaluated in descending order and rearranged;

[0012] Step 4: Auxiliary generation of text summaries based on contrastive learning;

[0013] Construct an optimization objective that makes the ROUGE-1 score ranking inversely proportional to the semantic similarity score;

[0014] A contrastive learning ranking loss function is designed, and a text summary of the source text is finally generated through contrastive learning.

[0015] Furthermore, the step 1 is specifically as follows:

[0016] The text embedding module consists of two parts, namely the word embedding layer and the position encoding embedding layer; the word embedding layer is used to dynamically embed the input text according to the context to obtain a dynamic word vector with context features, where the dimension of the word vector is d; the position encoding embedding layer is an encoding embedding with learnable parameters, and the current position of the input text is passed through the position encoding layer to obtain a position encoding vector, and the dimension of the position encoding vector is also d; for the source text embedding, the dynamic word vector X s Described by the following formula:

[0017] X s =PE(D)+WE(D) (1)

[0018] Among them, WE is word embedding, PE is positional encoding embedding, and D is the source text; the word embedding layer is obtained by the model during the pre-training process. Pre-training on the pre-training corpus enables the model to dynamically embed the input according to the context;

[0019] The source text encoding module consists of N layers of sub-encoders, each of which contains two sub-layers: the first-level sub-layer is a neural network based on the self-attention mechanism, and the second-level sub-layer is a fully connected feedforward neural network, which together constitute a complete sub-encoding system; within each sub-layer, residual connections are used to improve information flow, and layer normalization is used to ensure the stability of forward and backward propagation; in order to achieve residual connections between each sub-layer and each sub-encoder, the output of each sub-layer and sub-encoder of the model is a vector of dimension d; the source text encoding module extracts features from the input text, and obtains the context hidden layer vector after semantic encoding by multiple sub-encoders.

[0020] Preferably, the step 2 is specifically:

[0021] The context hidden layer vector obtained by the candidate summary generation model is used as the encoding hidden vector, a candidate summary set is generated according to the encoding hidden vector and the beam search algorithm, and a decoder is constructed for the encoding hidden vector to obtain the candidate summary set;

[0022] The decoder consists of N repeatedly stacked sub-decoders; each sub-decoder consists of four sub-layers, namely a masked self-attention network, two identical cross-attention networks, and a feed-forward neural network; in order to achieve residual connections within each sub-layer, the outputs of all sub-layers are vectors of dimension size d;

[0023] The masked self-attention network is used to decode and output the text information of the previous time step and perform forward encoding. Assuming that the current time step is t, the teacher forced training mechanism is first used to take the generation result of the previous time step (t-1) as the input of the current time step, and the dynamic word vector Y is obtained through the embedding layer. t-1 ; In this module, the text of the current time step can only focus on the text information of the previous time step. It is described by the following formula:

[0024] Y t-1 =LN(Y t-1 +MaskedSelfAttention(Y t-1 )) (2)

[0025] Among them, MaskedSelfAttention is a masked self-attention network, and LN is a layer normalization operation;

[0026] The cross-attention network allows the model to decode and generate output from the source text based on the calculation of the encoded latent vector; finally, the output is sent to the feedforward neural network for feature transformation and extraction to generate a candidate summary set C.

[0027] Preferably, the step 3 is specifically:

[0028] The candidate summary evaluation model uses the encoder network of the candidate summary generation model to encode and embed the candidate summary set C, source text D and reference summary R respectively to obtain the sentence vector encoding and It is specifically described by the following formula:

[0029]

[0030] in l is the total number of candidate summaries, is the maximum sequence length of the candidate summary, d is the hidden layer size;

[0031] After obtaining the sentence vector encoding of the candidate summary set, the ROUGE-1 score index of the candidate summary set C and the reference summary R is compared. Rearrange them in order from largest to smallest, as described by the following formula:

[0032]

[0033] in, Thus, we obtain the candidate summary sentence vectors arranged in order. Source text sentence vector and the reference summary sentence vector

[0034] Preferably, the step 4 is specifically:

[0035] When performing contrastive learning training, we first obtain the candidate summary sentence vectors arranged in sequence. Source text sentence vector and the reference summary sentence vector Their dimensions are all sequence length × hidden layer size;

[0036] After the first-last average pooling sentence vector operation, the candidate summary sentence vector Depend on Transform to Each candidate summary sentence vector Depend on Transform to Source text sentence vector Depend on Transform to Reference summary sentence vector Depend on Transform to

[0037] The designed contrastive learning ranking loss function is described by the following formula:

[0038]

[0039] Where i and j are the candidate summary set indexes; γ1, γ2 and γ3 are the parameters of the candidate summaries at different front and back positions, where γ1 = 20e (l-i)×0.1 ,γ2=20e (j-i)×0.1 ,γ3=20e (i-j)×0.1 , l is the total number of candidate summaries;

[0040] The loss function in formula (5) It is divided into three parts. The first part calculates the candidate summary. With source text and candidate summary Abstract with references The cosine similarity of I hope to refer to the abstract Compare to candidate summaries With source text The parameter γ1 is to impose a greater penalty on the candidate summaries with lower rankings. The second and third parts calculate the candidate summaries. With source text and candidate summary With source text The cosine similarity of Hope candidate summary Compare to candidate summaries With source text They are more semantically similar, and the parameters γ2 and γ3 are also intended to impose a greater penalty on the lower-ranked candidate summaries;

[0041] Based on this loss function, the semantic similarity of each candidate summary can be effectively evaluated according to the source text, and finally the most reasonable text summary of the source text can be given.

[0042] The beneficial effects of the present invention are as follows:

[0043] The present invention constructs a text summary generation framework including a candidate summary generation model and a candidate summary evaluation model. The generation model includes a text embedding module and a source text encoding module. Based on the self-attention structure, multiple candidate summaries are generated by setting different sampling parameters of the beam search algorithm. The evaluation model is based on the encoder structure of the generation model, and a new ranking loss function is designed based on the contrastive learning idea. Based on this loss function, the semantic similarity of each candidate summary can be effectively evaluated according to the source text. On the public Chinese summary dataset LCSTS, the ROUGE-1 score of the text summary generated by the present invention is improved by 2.16 compared with the literature method, and the semantic similarity indicator BERT-Score is improved by 2.12. In addition, the candidate summary generation model in the present invention is based on a sequence-to-sequence generation structure, which can be effectively extended to the current mainstream generation model based on a sequence-to-sequence generation structure, and a more excellent generated summary is obtained on its basis. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 The flowchart of the text summary auxiliary generation method based on contrastive learning of the present invention.

[0045] Figure 2 Schematic diagram of the structure of the candidate summary generation model of the present invention.

[0046] Figure 3 Schematic diagram of the structure of the candidate summary evaluation model of the present invention. DETAILED DESCRIPTION

[0047] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0048] Most existing deep learning-based text summarization methods tend to generate summaries with low diversity, while the candidate summaries with the highest degree of matching with the source text are hidden behind the high-probability candidate summaries, thereby weakening the semantic matching degree between the summary and the source text and affecting practical applications. In view of the above problems, the present invention discloses a text summarization auxiliary generation method based on contrastive learning.

[0049] like Figure 1-Figure 3 As shown, the technical solution of the present invention includes the following steps: candidate summary generation model construction, candidate summary set generation, candidate summary evaluation model construction and text summary generation based on contrastive learning.

[0050] (a) The present invention constructs a candidate summary generation model that can obtain a set of candidate summaries from a source text. The candidate summary generation model constructed by the present invention consists of two parts, namely a text embedding module and a source text encoding module. The text embedding module performs vocabulary segmentation, word embedding and position encoding embedding on the source text to obtain an embedding vector of the source text. The source text encoding module is responsible for semantic modeling and feature extraction of the source text.

[0051] (b) The present invention generates a candidate summary set based on the constructed candidate summary generation model according to the encoding latent vector and the beam search algorithm. The candidate summary set is obtained by constructing a decoder for the encoding latent vector. The decoder includes a masked self-attention network, a cross-attention network and a feedforward neural network. The masked self-attention network is used to decode and output the text information of the previous time step and perform forward encoding. The cross-attention network allows the model to decode and generate output from the source text based on the calculation of the encoding latent vector; finally, the output is sent to the feedforward neural network for feature transformation and extraction, thereby generating a candidate summary set.

[0052] (c) The present invention constructs a candidate summary evaluation model. The candidate summary evaluation model uses the encoder network of the candidate summary generation model to encode and embed the candidate summary set, the source text and the reference summary respectively to obtain the sentence vector encoding. Then, according to the similarity score index (ROUGE-1 score) between the candidate summary set and the reference summary, the candidate summaries are evaluated in descending order and rearranged.

[0053] (d) The present invention realizes the auxiliary generation of text summaries based on contrastive learning. First, an optimization goal is constructed. The optimization goal hopes that the model will give a higher semantic similarity score to the candidate summaries with higher ROUGE-1 scores, and give a lower semantic similarity score to the candidate summaries with lower ROUGE-1 scores. The candidate summary set and the source text are only arranged according to the degree of semantic similarity to a certain extent, and are not strictly sorted by semantic similarity. Furthermore, the present invention designs a loss function for contrastive learning sorting, that is, it hopes that the reference summary is more semantically similar to the source text than the candidate summary, and at the same time imposes a greater penalty on the candidate summaries with lower rankings. Through contrastive learning, the most reasonable text summary of the source text is finally generated. Specific embodiment:

[0055] 1) Candidate summary generation model construction

[0056] The candidate summary generation model consists of two parts: a text embedding module and a source text encoding module. The text embedding module performs vocabulary segmentation, word embedding, and position encoding embedding on the source text to obtain the embedding vector of the source text. The source text encoding module is responsible for semantic modeling and feature extraction of the source text.

[0057] The text embedding module consists of two parts, namely the word embedding layer and the position encoding embedding layer. The word embedding layer is used to dynamically embed the input text according to the context to obtain a dynamic word vector with context features, where the dimension of the word vector is d. The position encoding embedding layer is an encoding embedding with learnable parameters. The current position of the input text is passed through the position encoding layer to obtain a position encoding vector, and the dimension of the position encoding vector is also d. Taking the source text embedding as an example, the dynamic word vector X obtained by embedding in this paper is s It can be described by the following formula:

[0058] X s =PE(D)+WE(D) (1)

[0059] Among them, WE is word embedding, PE is positional encoding embedding, and D is the source text. The word embedding layer is obtained by the model during the pre-training process. Pre-training on a large-scale pre-training corpus enables the model to dynamically embed the input according to the context. Since the model does not contain any recursive and convolutional structures, in order for the model to learn the order relationship of the sequence, the model needs to learn to distinguish the word order of the previous and next texts during the pre-training process to ensure the position invariance and translation invariance of the sequence.

[0060] The source text encoding module consists of N layers of sub-encoders, each of which contains two sub-layers: the first level is a neural network based on the self-attention mechanism, and the second level is a fully connected feedforward neural network, which together constitute a complete sub-encoding system. Within each layer, residual connections are used to improve information flow, and layer normalization is used to ensure the stability of forward and backward propagation. In order to achieve residual connections between each sub-layer and each sub-encoder, the output of each sub-layer and sub-encoder of the model is a vector of dimension size d. The source text encoding module extracts features from the input text, and obtains the context hidden layer vector after semantic encoding by multiple sub-encoders.

[0061] 2) Generation of candidate summary set

[0062] The context hidden layer vector obtained by the candidate summary generation model is used as the encoding hidden vector, and a candidate summary set is generated based on the encoding hidden vector and the beam search algorithm. The candidate summary set is obtained by constructing a decoder for the encoding hidden vector.

[0063] Similar to the source text encoding module, the decoder also contains N repeatedly stacked sub-decoders. Each sub-decoder contains four sub-layers, namely a masked self-attention network, two identical cross-attention networks, and a feedforward neural network. In order to realize residual connections in each sub-layer, the outputs of all sub-layers are vectors of dimension size d.

[0064] The masked self-attention network is used to decode and output the text information of the previous time step and perform forward encoding. Assuming that the current time step is t, the teacher forced training mechanism is first used to take the generation result of the previous time step (t-1) as the input of the current time step, and the dynamic word vector Y is obtained through the embedding layer. t-1 In order to ensure that there is no information leakage during training and testing, the text of the current time step in this module can only focus on the text information of the previous time step. It can be described by the following formula:

[0065] Y t-1 =LN(Y t-1 +MaskedSelfAttention(Y t-1 )) (2)

[0066] Among them, MaskedSelfAttention is a masked self-attention network, and LN is a layer normalization operation.

[0067] The cross-attention network allows the model to decode and generate output from the source text based on the calculation of the encoded latent vector; finally, the output is sent to the feedforward neural network for feature transformation and extraction to generate a candidate summary set C.

[0068] 3) Construction of candidate summary evaluation model

[0069] The candidate summary evaluation model uses the encoder network of the candidate summary generation model to encode and embed the candidate summary set C, source text D and reference summary R respectively to obtain the sentence vector encoding and The specific description can be as follows:

[0070]

[0071] in l is the total number of candidate summaries, is the maximum sequence length of the candidate summary, and d is the hidden layer size. After obtaining the sentence vector encoding of the candidate summary set, it is necessary to compare the candidate summary set C with the reference summary R according to the ROUGE-1 score index. Rearrange in order from largest to smallest, which can be described by the following formula:

[0072]

[0073] in, Thus, the candidate summary sentence vectors arranged in sequence are obtained Source text sentence vector and the reference summary sentence vector

[0074] 4) Text summarization assisted generation based on contrastive learning

[0075] The most important thing before training the candidate summary evaluation model is to generate sentence vectors. From the above analysis, we know that when performing contrastive learning training, we first obtain the candidate summary sentence vectors arranged in order. Source text sentence vector and the reference summary sentence vector Their dimensions are all sequence length × hidden layer size. However, due to the inconsistency of sequence lengths between the vectors, the similarity cannot be calculated, so the dimensions need to be unified. The first-last average pooled sentence vector is to average the encoding vectors of all characters in the first and last layers of the encoder and then perform summing and pooling operations to obtain the final semantic encoding representation.

[0076] After the first-last average pooling sentence vector operation, the candidate summary sentence vector Depend on Transform to Each candidate summary sentence vector Depend on Transform to Source text sentence vector Depend on Transform to Reference summary sentence vector Depend on Transform to

[0077] The contrastive learning ranking loss function designed by the present invention can be described by the following formula:

[0078]

[0079] Where i and j are the candidate summary set indexes. γ1, γ2 and γ3 are the parameters of the candidate summaries at different front and back positions. Where γ1 = 20e (l-i)×0.1 ,γ2=20e (j-i)×0.1 ,γ3=20e (i-j)×0.1 , l is the total number of candidate summaries. From (5), we can see that the loss function It is divided into three parts. The first part calculates the candidate summary With source text and candidate summary Abstract with references The cosine similarity of I hope to refer to the abstract Compare to candidate summaries With source text The parameter γ1 is to impose a greater penalty on the candidate summaries with lower rankings. The second and third parts calculate the candidate summaries With source text and candidate summary With source text The cosine similarity of Hope candidate summary Compare to candidate summaries With source text The parameters γ2 and γ3 are also intended to impose a larger penalty on the lower-ranked candidate summaries.

[0080] Based on this loss function, the semantic similarity of each candidate summary can be effectively evaluated according to the source text, and finally the most reasonable text summary of the source text can be given.

Claims

1. A text summary auxiliary generation method based on contrastive learning, characterized in that: The steps include: Step 1: Build a candidate summary generation model; The candidate summary generation model constructed consists of two parts, namely, a text embedding module and a source text encoding module. The text embedding module performs vocabulary segmentation, word embedding and position encoding embedding on the source text to obtain the embedding vector of the source text. The source text encoding module is responsible for semantic modeling and feature extraction of the source text. Step 2: Based on the constructed candidate summary generation model, a candidate summary set is generated according to the encoding latent vector and the beam search algorithm; The decoder is constructed for the encoded latent vector to obtain a set of candidate summaries; the decoder includes a masked self-attention network, a cross-attention network, and a feedforward neural network; the masked self-attention network is used to decode and output the text information of the previous time step and perform forward encoding; the cross-attention network allows the model to decode and generate output from the source text based on the calculation of the encoded latent vector; finally, the output is sent to the feedforward neural network for feature transformation and extraction, thereby generating a set of candidate summaries; Step 3: Build a candidate summary evaluation model; The candidate summary evaluation model uses the encoder network of the candidate summary generation model to encode and embed the candidate summary set, source text and reference summary respectively to obtain sentence vector encoding; then, according to the similarity score indicator between the candidate summary set and the reference summary, namely the ROUGE-1 score, the candidate summaries are evaluated in descending order and rearranged; Step 4: Auxiliary generation of text summaries based on contrastive learning; Construct an optimization objective that makes the ROUGE-1 score ranking inversely proportional to the semantic similarity score; A contrastive learning ranking loss function is designed, and a text summary of the source text is finally generated through contrastive learning.

2. The text summary auxiliary generation method based on contrastive learning according to claim 1 is characterized in that: The step 1 is specifically as follows: The text embedding module consists of two parts, namely the word embedding layer and the position encoding embedding layer; the word embedding layer is used to dynamically embed the input text according to the context to obtain a dynamic word vector with context features, where the dimension of the word vector is d; the position encoding embedding layer is an encoding embedding with learnable parameters, and the current position of the input text is passed through the position encoding layer to obtain a position encoding vector, and the dimension of the position encoding vector is also d; for the source text embedding, the dynamic word vector X s Described by the following formula: X s =PE(D)+WE(D) (1) Among them, WE is word embedding, PE is positional encoding embedding, and D is the source text; the word embedding layer is obtained by the model during the pre-training process. Pre-training on the pre-training corpus enables the model to dynamically embed the input according to the context; The source text encoding module consists of N layers of sub-encoders, each of which contains two sub-layers: the first-level sub-layer is a neural network based on the self-attention mechanism, and the second-level sub-layer is a fully connected feedforward neural network, which together constitute a complete sub-encoding system; within each sub-layer, residual connections are used to improve information flow, and layer normalization is used to ensure the stability of forward and backward propagation; in order to achieve residual connections between each sub-layer and each sub-encoder, the output of each sub-layer and sub-encoder of the model is a vector of dimension d; the source text encoding module extracts features from the input text, and obtains the context hidden layer vector after semantic encoding by multiple sub-encoders.

3. The text summary auxiliary generation method based on contrastive learning according to claim 2 is characterized in that: The step 2 is specifically as follows: The context hidden layer vector obtained by the candidate summary generation model is used as the encoding hidden vector, a candidate summary set is generated according to the encoding hidden vector and the beam search algorithm, and a decoder is constructed for the encoding hidden vector to obtain the candidate summary set; The decoder consists of N repeatedly stacked sub-decoders; each sub-decoder consists of four sub-layers, namely a masked self-attention network, two identical cross-attention networks, and a feed-forward neural network; in order to achieve residual connections within each sub-layer, the outputs of all sub-layers are vectors of dimension size d; The masked self-attention network is used to decode and output the text information of the previous time step and perform forward encoding. Assuming that the current time step is t, the teacher forced training mechanism is first used to take the generation result of the previous time step (t-1) as the input of the current time step, and the dynamic word vector Y is obtained through the embedding layer. t-1 ; In this module, the text of the current time step can only focus on the text information of the previous time step; it is specifically described by the following formula: AND t-1 =LN(Y t-1 +MaskedSelfAttention(Y t-1 )) (2) Among them, MaskedSelfAttention is a masked self-attention network, and LN is a layer normalization operation; The cross-attention network allows the model to decode and generate output from the source text based on the calculation of the encoded latent vector; finally, the output is sent to the feedforward neural network for feature transformation and extraction to generate a candidate summary set C.

4. The text summary auxiliary generation method based on contrastive learning according to claim 3 is characterized in that: The step 3 is specifically as follows: The candidate summary evaluation model uses the encoder network of the candidate summary generation model to encode and embed the candidate summary set C, source text D and reference summary R respectively to obtain the sentence vector encoding and It is specifically described by the following formula: in l is the total number of candidate summaries, is the maximum sequence length of the candidate summary, d is the hidden layer size; After obtaining the sentence vector encoding of the candidate summary set, the ROUGE-1 score index of the candidate summary set C and the reference summary R is compared. Rearrange them in order from largest to smallest, as described by the following formula: in, Thus, we obtain the candidate summary sentence vectors arranged in order. Source text sentence vector and the reference summary sentence vector 5. The text summary auxiliary generation method based on contrastive learning according to claim 4 is characterized in that: The step 4 is specifically as follows: When performing contrastive learning training, we first obtain the candidate summary sentence vectors arranged in sequence. Source text sentence vector and the reference summary sentence vector Their dimensions are all sequence length × hidden layer size; After the first-last average pooling sentence vector operation, the candidate summary sentence vector Depend on Transform to Each candidate summary sentence vector Depend on Transform to Source text sentence vector Depend on Transform to Reference summary sentence vector Depend on Transform to The designed contrastive learning ranking loss function is described by the following formula: Where i and j are the candidate summary set indexes; γ1, γ2 and γ3 are the parameters of the candidate summaries at different front and back positions, where γ1 = 20e (l-i)×0.1 ,γ2=20e (j-i)×0.1 ,γ3=20e (i-j)×0.1 , l is the total number of candidate summaries; The loss function in formula (5) It is divided into three parts. The first part calculates the candidate summary. With source text and candidate summary Abstract with references The cosine similarity of I hope to refer to the abstract Compare to candidate summaries With source text The parameter γ1 is to impose a greater penalty on the candidate summaries with lower rankings. The second and third parts calculate the candidate summaries. With source text and candidate summary With source text The cosine similarity of Hope candidate summary Compare to candidate summaries With source text They are more semantically similar, and the parameters γ2 and γ3 are also intended to impose a greater penalty on the lower-ranked candidate summaries; Based on this loss function, the semantic similarity of each candidate summary can be effectively evaluated according to the source text, and finally the most reasonable text summary of the source text can be given.

Images