Text summary generation method and system based on reinforcement learning

Abstract

The present invention provides a text summary generation method and system based on reinforcement learning, and relates to the technical field of automatic text summary generation. The present invention obtains and preprocesses the original text and standard summary of the source text; obtains a pre-trained model and a summary sequence based on the original text in the pre-processed source text and a seq2seq model; samples the summary sequence through the pre-trained model to obtain a sampling sequence; samples the standard summary through a greedy algorithm to obtain a standard sequence; calculates the loss function through the sampling sequence and the standard sequence, updates the parameters of the pre-trained model, repeats the above steps until the loss function converges, and obtains a summary generation model. When training the model, the present invention fully considers the losses of the ROUGE-1 indicator, the ROUGE-2 indicator, and the ROUGE-L indicator, thereby avoiding unnecessary repetition and content richness problems when using the summary generation model to generate a summary, and the generated summary has better fluency and readability and higher quality.

Description

Technical Field

[0001] The present invention relates to the technical field of automatic text summary generation, and in particular to a text summary generation method and system based on reinforcement learning. Background Art

[0002] Text summarization technology includes extractive text summarization technology and generative text summarization technology. Among them, extractive text summarization technology extracts important sentences that can summarize the information of the paper by extracting the first three lines of the text or using methods such as TextRank graphs and clustering. Although this technology has a high readability of the extracted language, it cannot highly summarize a sentence, which will cause the generated summary to include a lot of unimportant information and the generated text to be too redundant; generative text summarization technology allows the machine to learn the human way of thinking, use various deformation methods of neural networks to train the model, and learn the text content to let the machine generate some new words at each time step, making the generated summary more concise and concise.

[0003] The existing methods mainly include using reinforcement learning mechanism combined with other algorithms to generate text summaries, such as mixing Teacher-forcing and reinforcement learning policy gradient algorithm. During the training process, this method first uses the cross entropy loss function to train for several epochs, and then slowly switches to the reinforcement learning loss function for training. There are many ways to switch from the cross entropy loss function to the reinforcement learning loss function. For example, the combination of the two in the following formula: L mixed =μL r1 +(1-μ)L ce , where μ∈(0, 1) is a parameter that controls the transition from cross entropy loss to reinforcement learning loss. At the beginning of training, μ=0, and the model completely relies on cross entropy loss. As training progresses, the value of μ gradually increases to slowly reduce the impact of cross entropy loss.

[0004] However, using a hybrid training method can speed up the convergence of the training model and improve the performance of the model. However, during the training process, when the model switches from cross entropy loss to reinforcement learning loss, the content generated by the reinforcement learning algorithm will eventually have duplication and readability problems. Using the formula: L mixed =μL rl +(1-μ)L ce Although training can alleviate the problems of content duplication and readability, the improvement in model performance is limited, resulting in low readability of the generated summaries. Summary of the invention

[0005] 1. Technical issues to be solved

[0006] In view of the deficiencies in the prior art, the present invention provides a text summary generation method and system based on reinforcement learning, which solves the technical problem that the summary text generated by the prior method has low readability.

[0007] (II) Technical solution

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0009] In a first aspect, the present invention provides a method for generating a text summary based on reinforcement learning, the method comprising:

[0010] S1. Obtain and preprocess the original text and standard summary of the source text;

[0011] S2, obtaining a pre-trained model and summary sequence based on the pre-processed source text and seq2seq model;

[0012] S3, sampling the summary sequence through the pre-trained model to obtain a sampled sequence; sampling the standard summary through the greedy algorithm to obtain a standard sequence;

[0013] S4. Calculate the loss function through the sampling sequence and the standard sequence, update the parameters of the pre-trained model, repeat steps S2 to S4 until the loss function converges, and obtain a summary generation model. The summary generation model is used to automatically generate text summaries. The loss function includes the losses of the ROUGE-1 indicator, the ROUGE-2 indicator, and the ROUGE-L indicator.

[0014] Preferably, the step of obtaining a pre-trained model and a summary sequence based on the pre-processed source text and the seq2seq model includes:

[0015] S201. Obtaining a pre-trained model based on the Teacher-forcing mechanism and seq2seq model

[0016] S202. Pre-training model The preprocessed original text is processed to obtain a summary sequence.

[0017] Preferably, the seq2seq model is an attention-based pointer generation seq2seq model.

[0018] Preferably, the sampling of the summary sequence by the pre-trained model to obtain the sampled sequence; and the sampling of the standard summary by the greedy algorithm to obtain the standard sequence include:

[0019] Leveraging pre-trained models Sample a specific action from each summary sequence Using a greedy algorithm, select actions from the standard summary

[0020] Through pre-trained model Action Sampling is performed and the action is sampled through a greedy algorithm Sampling is performed to obtain the sampling sequence and standard sequence

[0021] Preferably, the calculation formula of the loss function includes:

[0022]

[0023]

[0024] Among them, the formula Represents the sampling sequence ROUGE-1, ROUGE-2, and ROUOGE-L scores were calculated separately; Represents the standard sequence ROUGE-1, ROUGE-2, and ROUGE-L scores are calculated respectively; λμσ represent weight coefficients respectively.

[0025] Preferably, the method further comprises:

[0026] The parameters of the pre-trained model are trained in combination with the joint loss function, and the calculation formula of the joint loss function is as follows:

[0027] L mixed =μL rl +(1-μ)L ce

[0028] Among them, μ∈(0,1) is a parameter that controls the transition from cross entropy loss to reinforcement learning loss. At the beginning of training, μ=0, and as the training progresses, the value of μ gradually increases; L rl is the reinforcement learning loss, L ce is the cross entropy loss.

[0029] In a second aspect, the present invention provides a text summary generation system based on reinforcement learning, the system comprising:

[0030] A data processing module, used for acquiring and preprocessing the original text and the standard summary of the source text;

[0031] A synthesis module for obtaining pre-trained models and summary sequences based on pre-processed source text and seq2seq models;

[0032] The sampling module is used to sample the summary sequence through the pre-trained model to obtain the sampled sequence; and to sample the standard summary through the greedy algorithm to obtain the standard sequence;

[0033] The summary generation model acquisition module is used to calculate the loss function through the sampling sequence and the standard sequence, update the parameters of the pre-trained model, repeat the execution process in the comprehensive module and the sampling module until the loss function converges, and obtain the summary generation model. The summary generation model is used to automatically generate text summaries. The loss function includes the losses of the ROUGE-1 indicator, the ROUGE-2 indicator and the ROUGE-L indicator.

[0034] Preferably, the comprehensive module includes:

[0035] Pre-training module, used to obtain pre-trained models based on the Teacher-forcing mechanism and seq2seq model

[0036] Summary sequence acquisition module, used to obtain the pre-trained model The preprocessed original text is processed to obtain a summary sequence.

[0037] In a third aspect, the present invention provides a computer-readable storage medium, characterized in that it stores a computer program for generating text summaries based on reinforcement learning, wherein the computer program enables a computer to execute the text summaries generating method based on reinforcement learning as described above.

[0038] In a fourth aspect, the present invention provides an electronic device, characterized in that it includes:

[0039] one or more processors;

[0040] Memory; and

[0041] One or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include instructions for executing the reinforcement learning-based text summary generation method as described above.

[0042] (III) Beneficial effects

[0043] The present invention provides a method and system for generating text summaries based on reinforcement learning. Compared with the prior art, it has the following beneficial effects:

[0044] The present invention obtains and preprocesses the original text and standard summary of the source text; obtains a pretrained model and a summary sequence based on the original text in the preprocessed source text and a seq2seq model; samples the summary sequence through the pretrained model to obtain a sampled sequence; samples the standard summary through a greedy algorithm to obtain a standard sequence; calculates a loss function through the sampled sequence and the standard sequence, updates the parameters of the pretrained model, and repeats the above steps until the loss function converges to obtain a summary generation model, the summary generation model is used to automatically generate text summaries, and the loss function includes the loss of ROUGE-1 index, ROUGE-2 index and ROUGE-L index. When training the model, the present invention fully considers the loss of ROUGE-1 index, ROUGE-2 index and ROUGE-L index, thereby avoiding unnecessary repetition and content richness problems when using the summary generation model to generate summaries, and the generated summaries are more fluent and readable, and of higher quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 The block diagram is a method for generating text summaries based on reinforcement learning in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] The embodiments of the present application provide a method and system for generating text summaries based on reinforcement learning, thereby solving the technical problem that summaries generated by existing methods have low readability and achieving summaries with high readability.

[0049] The technical solution in the embodiment of the present application is to solve the above technical problems, and the overall idea is as follows:

[0050] In the automatic text summarization task, ROUGE-1 (unigram) and ROUGE-2 (bigram) are used to measure the richness of summary information, and ROUGE-L represents the longest common subsequence of the reference summary and the candidate summary, which is used to measure the fluency of the summary content. Therefore, the embodiment of the present invention proposes a new reinforcement learning loss function for model training. By comprehensively considering the scores of all ROUGE indicators as rewards to train the model, different weights are assigned to different ROUGE indicators, so that the model is more inclined during the training process (for example, the ROUGE-1 indicator and the ROUGE-L indicator are considered with the same weight), so that the model can ensure both the richness and fluency of the generated content when training to generate summaries, thereby solving the problems existing in previous reinforcement learning methods.

[0051] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0052] The embodiment of the present invention provides a method and system for generating a text summary based on reinforcement learning, the method comprising the following steps:

[0053] S1. Obtain and preprocess the original text and standard summary of the source text;

[0054] S2, obtaining a pre-trained model and summary sequence based on the pre-processed source text and seq2seq model;

[0055] S3, sampling the summary sequence through the pre-trained model to obtain a sampled sequence; sampling the standard summary through the greedy algorithm to obtain a standard sequence;

[0056] S4. Calculate the loss function through the sampling sequence and the standard sequence, update the parameters of the pre-trained model, repeat steps S2 to S4 until the loss function converges, and obtain a summary generation model. The summary generation model is used to automatically generate text summaries. The loss function includes the losses of the ROUGE-1 indicator, the ROUGE-2 indicator, and the ROUGE-L indicator.

[0057] When training the model, the embodiment of the present invention fully considers the losses of the ROUGE-1 indicator, the ROUGE-2 indicator and the ROUGE-L indicator, thereby avoiding unnecessary repetition and content richness problems when generating summaries using the summary generation model. The generated summaries are more fluent and readable, and of higher quality.

[0058] The specific implementation process of the embodiment of the present invention is described in detail below:

[0059] In step S1, the original text and the standard summary of the source text are obtained and preprocessed. The specific implementation process is as follows:

[0060] The computer obtains the source text, and processes the original text and the standard summary in the source text respectively to construct a vocabulary. The function of the vocabulary is to reflect the mapping relationship between the generated words and the word IDs.

[0061] In step S2, a pre-trained model and a summary sequence are obtained based on the pre-processed source text and the seq2seq model. The specific implementation process is as follows:

[0062] S201. Obtaining a pre-trained model based on the Teacher-forcing mechanism and seq2seq model Specifically include:

[0063] Input an original text X and its corresponding ground-truth summary Y (i.e., the standard summary in the corresponding original text) in the preprocessed source text into the seq2seq model, and train it through the cross entropy loss function to obtain the pre-trained model

[0064] In the pre-training process, the cross entropy loss function is used to train a better baseline, and then the reinforcement learning method is used for training. This will allow the sampling space to have better results and speed up the convergence of the training model.

[0065] S202. Pre-training model The preprocessed original text is processed to obtain a summary sequence; specifically, the process includes:

[0066] In this embodiment of the present invention, the pre-trained model You can choose between an attention-based seq2seq model and an attention-based pointer-generated seq2seq model.

[0067] The following describes in detail the process of obtaining summary sequences by using two models: an attention-based seq2seq model and an attention-based pointer generation seq2seq model.

[0068] When using the attention-based seq2seq model:

[0069] After the input is converted into the corresponding word vector, the original word vector is encoded into an intermediate latent vector h through the encoder i , the decoder inputs the reference summary word vectors in sequence and obtains the decoding state s at time step t t , use h i and t Calculate the attention weight of the i-th word in the original text at this time step

[0070]

[0071]

[0072] Then use the obtained attention vector and the latent vector h i Calculate the context vector

[0073]

[0074] in, It can be seen as a vector representation of the original text, and then the decoding state s is obtained by taking the time step t t and After concatenation through the linear layer, the final word list distribution P is obtained vocab .

[0075]

[0076] In this way, a probability distribution is obtained through softmax, and the words that need to be generated can be predicted.

[0077] P(w)=P vocab (w)

[0078] After all decoding steps are completed, the summary sequence is obtained.

[0079] When using attention-based pointers to generate seq2seq models:

[0080] In the basic seq2seq+attention model, a new weight P is introduced gen To weigh whether a word should be generated or copied. Get the context vector from the basic attention seq2seq model and the state vector s t After that, combined with the decoder input x t Calculate P together gen ,

[0081]

[0082] At this time, it will be expanded into a larger word list (including the words in the original text).

[0083]

[0084] The subsequent steps are consistent with the attention-based seq2seq model steps. After all decoding steps are completed, the summary sequence is obtained.

[0085] In step S3, the summary sequence is sampled by the pre-trained model to obtain a sampled sequence; the standard summary is sampled by the greedy algorithm to obtain a standard sequence. The specific implementation process is as follows:

[0086] Leveraging pre-trained models Sample a specific action from each summary sequence

[0087] Using the greedy algorithm (denoted as π g ), select actions from the standard summary Through pre-trained model Action Sampling is performed and the action is sampled through a greedy algorithm Sampling is performed to obtain the sampling sequence and standard sequence The final sampling sequence is: as well as

[0088] In step S4, the loss function is calculated by the sampling sequence and the standard sequence, the parameters of the pre-trained model are updated, and steps S2 to S4 are repeated until the loss function converges to obtain a summary generation model, which is used to automatically generate text summaries. The loss function includes the losses of the ROUGE-1 indicator, the ROUGE-2 indicator, and the ROUGE-L indicator. The specific implementation process is as follows:

[0089] According to the sampling sequence and the standard sequence, the ROUGE scores (including ROUGE-1, ROUGE-2, and ROUGE-L) are calculated:

[0090]

[0091]

[0092] in, They represent the sampling results of each time step of the decoder in the pre-trained model respectively, and the whole represents a complete sequence; They represent the sampling results of each time step of the greedy algorithm respectively, and the whole represents a complete sequence;

[0093] In the formula Represents the sampling sequence ROUGE-1, ROUGE-2, and ROUOGE-L scores were calculated separately; Represents the standard sequence ROUGE-1, ROUGE-2, and ROUGE-L scores are calculated respectively; λμσ represent weight coefficients respectively.

[0094] The loss function is calculated by the above formula to update the model parameters. The trained model is recorded as π θ The weight parameters are set at the beginning of training, for example, λ = 0.5, μ = 0, σ = 0.5, which means that the ROUGE-1 score and the ROUGE-L score are considered with the same weight, constraining the richness and fluency of the generated summary content.

[0095] In the specific implementation process, the parameters of the pre-trained model can also be trained by combining the joint loss function, and the calculation formula of the joint loss function is as follows:

[0096] L mixed =μL rl +(1-μ)L ce

[0097] Among them, μ∈(0, 1) is a parameter that controls the transition from cross entropy loss to reinforcement learning loss. At the beginning of training, μ=0, and the model completely relies on cross entropy loss. As the training progresses, the value of μ gradually increases to slowly reduce the impact of cross entropy loss. L rl is the reinforcement learning loss, L ce is the cross entropy loss.

[0098] The summary generation model processes the original text for generating the summary as follows:

[0099] First, process the original text to generate the summary and obtain the word vector.

[0100] Input the word vector into the summary generation model and output the summary.

[0101] The method proposed in the embodiment of the present invention can train the model faster under the same conditions. After experiments on relevant data sets (for example, the parameters set are λ=0.5, μ=0, σ=0.5), compared with the traditional reinforcement learning method, the embodiment of the present invention can continue to improve the performance of the model and generate a more readable summary.

[0102] In the experiment, the English text summary dataset gigaword is selected for the experiment:

[0103] 1) The results obtained by training only with the cross entropy loss function:

[0104] ROUGE-1 ROUGE-2 ROUGE-L 0.4412 0.2323 0.4047

[0105] 2) The results obtained by pre-training with cross entropy loss and then training with the traditional RL loss function (λ=0, μ=0, σ=1):

[0106] ROUGE-1 ROUGE-2 ROUGE-L 0.3456 0.1770 0.4540

[0107] The ROUGE-L score is higher than the ROUGE-1 score, indicating that the generated content has a lot of repetition (that is, the generated summary tends to stutter).

[0108] 3) The results obtained by combining the cross entropy loss function and the RL loss function:

[0109] ROUGE-1 ROUGE-2 ROUGE-L 0.4499 0.2403 0.4132

[0110] 4) Results obtained using the solution proposed in the embodiment of the present invention:

[0111]

[0112]

[0113] An embodiment of the present invention provides a text summary generation system based on reinforcement learning, the system comprising:

[0114] A data processing module, used for acquiring and preprocessing the original text and the standard summary of the source text;

[0115] A synthesis module for obtaining pre-trained models and summary sequences based on pre-processed source text and seq2seq models;

[0116] The sampling module is used to sample the summary sequence through the pre-trained model to obtain the sampled sequence; and to sample the standard summary through the greedy algorithm to obtain the standard sequence;

[0117] The summary generation model acquisition module is used to calculate the loss function through the sampling sequence and the standard sequence, update the parameters of the pre-trained model, repeat the execution process in the comprehensive module and the sampling module until the loss function converges, and obtain the summary generation model. The summary generation model is used to automatically generate text summaries. The loss function includes the losses of the ROUGE-1 indicator, the ROUGE-2 indicator and the ROUGE-L indicator.

[0118] An embodiment of the present invention also provides a storage medium storing a computer program for generating text summaries based on reinforcement learning, wherein the computer program enables a computer to execute the text summaries generating method based on reinforcement learning as described above.

[0119] An embodiment of the present invention further provides an electronic device, including:

[0120] one or more processors;

[0121] Memory; and

[0122] One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs including instructions for executing the reinforcement learning-based text summary generation method as described above.

[0123] It can be understood that the reinforcement learning-based text summary generation system, storage medium, and electronic device provided in the embodiments of the present invention correspond to the reinforcement learning-based text summary generation method provided in the embodiments of the present invention. The explanations, examples, and beneficial effects of the relevant contents can refer to the corresponding parts in the reinforcement learning-based text summary generation method, and will not be repeated here.

[0124] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0125] When training the model, the embodiment of the present invention fully considers the losses of the ROUGE-1 indicator, the ROUGE-2 indicator and the ROUGE-L indicator, thereby avoiding unnecessary repetition and content richness problems when generating summaries using the summary generation model. The generated summaries are more fluent and readable, and of higher quality.

[0126] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0127] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A text summary generation method based on reinforcement learning, characterized in that: The method comprises: S1. Obtain and preprocess the original text and standard summary of the source text; S2, obtaining a pre-trained model and summary sequence based on the pre-processed source text and seq2seq model; S3, sampling the summary sequence through the pre-trained model to obtain a sampled sequence; sampling the standard summary through the greedy algorithm to obtain a standard sequence; S4, calculating the loss function through the sampling sequence and the standard sequence, updating the parameters of the pre-trained model, repeating steps S2 to S4 until the loss function converges, and obtaining a summary generation model, wherein the summary generation model is used to automatically generate a text summary, and the loss function includes the losses of the ROUGE-1 indicator, the ROUGE-2 indicator, and the ROUGE-L indicator; Among them, obtaining and preprocessing the original text and standard summary of the source text includes: The original text and the standard summary in the source text are obtained and processed respectively to construct a vocabulary; the vocabulary is used to reflect the mapping relationship between the generated words and the IDs of the words; The specific implementation process of obtaining a pre-trained model and a summary sequence based on the pre-processed source text and the seq2seq model is as follows: S201. Obtaining a pre-trained model based on the Teacher-forcing mechanism and seq2seq model ; Specifically include: Input an original text X and its corresponding standard summary Y from the preprocessed source text into the seq2seq model, and train it through the cross entropy loss function to obtain a pre-trained model S202. Pre-training model Process the preprocessed original text to obtain a summary sequence; The method of sampling the summary sequence by the pre-trained model to obtain the sampled sequence and sampling the standard summary by the greedy algorithm to obtain the standard sequence includes: Leveraging pre-trained models Sample a specific action from each summary sequence Using a greedy algorithm, select actions from the standard summary Through pre-trained model Action Sampling is performed and the action is sampled through a greedy algorithm Sampling is performed to obtain the sampling sequence and standard sequence S201. Obtaining a pre-trained model based on the Teacher-forcing mechanism and seq2seq model S202. Pre-training model The preprocessed original text is processed to obtain a summary sequence.

2. The text summary generation method based on reinforcement learning as claimed in claim 1, characterized in that: The seq2seq model is an attention-based pointer generation seq2seq model.

3. The text summary generation method based on reinforcement learning as claimed in claim 1, characterized in that: The calculation formula of the loss function includes: Among them, the formula Represents the sampling sequence ROUGE-1, ROUGE-2, and ROUOGE-L scores were calculated separately; Represents the standard sequence ROUGE-1, ROUGE-2, and ROUGE-L scores are calculated respectively; λμσ represent weight coefficients respectively.

4. The text summary generation method based on reinforcement learning as claimed in claim 3, characterized in that: The method further comprises: The parameters of the pre-trained model are trained in combination with the joint loss function, and the calculation formula of the joint loss function is as follows: L mixed =μL rl +(1-μ)L ce Among them, μ∈(0,1) is a parameter that controls the transition from cross entropy loss to reinforcement learning loss. At the beginning of training, μ=0, and as the training progresses, the value of μ gradually increases; L rl is the reinforcement learning loss, L ce is the cross entropy loss.

5. A text summary generation system based on reinforcement learning, characterized in that: The system comprises: A data processing module, used for acquiring and preprocessing the original text and the standard summary of the source text; A synthesis module for obtaining pre-trained models and summary sequences based on pre-processed source text and seq2seq models; The sampling module is used to sample the summary sequence through the pre-trained model to obtain the sampled sequence; and to sample the standard summary through the greedy algorithm to obtain the standard sequence; A summary generation model acquisition module is used to calculate the loss function through the sampling sequence and the standard sequence, update the parameters of the pre-trained model, repeat the execution process in the comprehensive module and the sampling module until the loss function converges, and obtain the summary generation model, wherein the summary generation model is used to automatically generate text summaries, and the loss function includes the losses of the ROUGE-1 indicator, the ROUGE-2 indicator, and the ROUGE-L indicator; Among them, obtaining and preprocessing the original text and standard summary of the source text includes: The original text and the standard summary in the source text are obtained and processed respectively to construct a vocabulary; the vocabulary is used to reflect the mapping relationship between the generated words and the IDs of the words; The specific implementation process of obtaining a pre-trained model and a summary sequence based on the pre-processed source text and the seq2seq model is as follows: S201. Obtaining a pre-trained model based on the Teacher-forcing mechanism and seq2seq model ; Specifically include: Input an original text X and its corresponding standard summary Y from the preprocessed source text into the seq2seq model, and train it through the cross entropy loss function to obtain a pre-trained model S202. Pre-training model Process the preprocessed original text to obtain a summary sequence; The method of sampling the summary sequence by the pre-trained model to obtain the sampled sequence and sampling the standard summary by the greedy algorithm to obtain the standard sequence includes: Leveraging pre-trained models Sample a specific action from each summary sequence Using a greedy algorithm, select actions from the standard summary Through pre-trained model Action Sampling is performed and the action is sampled through a greedy algorithm Sampling is performed to obtain the sampling sequence and standard sequence 6. A computer-readable storage medium, characterized in that: It stores a computer program for generating text summaries based on reinforcement learning, wherein the computer program enables a computer to execute the method for generating text summaries based on reinforcement learning as claimed in any one of claims 1 to 4.

7. An electronic device, characterized in that: include: one or more processors; Memory; as well as One or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include a program for executing the reinforcement learning-based text summary generation method according to any one of claims 1 to 4.

Images