Text summarization generation model training methods, related equipment, and readable storage media

Abstract

The present application discloses a text summary generation model training method, related equipment and readable storage medium. When the above method is used to train the text summary generation model, since the idea of ​​remote supervision is used to determine the summary label, text category label and text style label of the training text, there is no need to rely on manually annotated data. The text summary generation model composed of the encoding layer, the interaction layer, the text summary automatic generation layer, the text category adaptation layer and the text style adaptation layer can solve the polysemy and gradient disappearance problems of the sequence to sequence model based on the long short-term memory network. Moreover, the influence of text category and text style on the summary content is considered during training. Therefore, the text summary generation model obtained by the above training is used to automatically generate a text summary, which can improve the accuracy of the generated text summary.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and more specifically, to a text summary generation model training method, related equipment and readable storage medium. Background Art

[0002] With the advent of the Internet information age, people can receive more and more information, but people usually have limited time to process text data. How to receive more meaningful information in a limited time, text summarization is a good solution. Considering that manual summary generation of long texts is a very time-consuming and laborious task, automatic text summary generation is particularly necessary.

[0003] Automatic text summarization is to extract the important theme from a long text and generate a relatively short text that can express the theme. It can be divided into extractive and generative methods. The extractive method extracts key sentences and keywords from a long text to form a summary. The summary content is all from the original text. There is a certain guarantee in grammar and syntax, but there may be problems such as incorrect content selection, poor coherence, and poor flexibility. Generative summarization generates summaries word by word based on long texts. It allows the generation of new words and phrases and has high flexibility. With the development of neural network models, text summary generation models are often used for generative summary tasks. At present, the sequence-to-sequence (Seq2Seq) model based on the long short-term memory network (LSTM) is mostly used as a text summary generation model.

[0004] The Sequence to Sequence (Seq2Seq) model based on the Long Short-Term Memory (LSTM) network is trained using a supervised method and relies on a large amount of manually annotated data, but in reality the amount of manually annotated data is often insufficient. Moreover, the model uses a fixed word vector as the expression of each word, which cannot solve the problem of polysemy. In addition, the Long Short-Term Memory (LSTM) network will have a gradient vanishing problem for long sentences. In addition, the summary content of texts of different categories and different language styles will also be different, and the model does not take into account the impact of the above differences on the summary content. The accuracy of the text summary generated by this model is low.

[0005] Therefore, how to improve the accuracy of the generated text summary has become a technical problem that needs to be solved urgently by those skilled in the art. Summary of the invention

[0006] In view of the above problems, this application proposes a text summary generation model training method, related equipment and readable storage medium. The specific scheme is as follows:

[0007] A text summary generation model training method, the method comprising:

[0008] Obtaining a pre-built text summary generation model, wherein the text summary generation model includes an encoding layer, an interaction layer, a text summary automatic generation layer, a text category adaptation layer, and a text style adaptation layer;

[0009] Get training text;

[0010] Based on the idea of ​​distant supervision, determining the summary label, text category label and text style label of the training text;

[0011] The training text is used as a training sample, the summary label, text category label and text style label of the training text are used as sample labels, and the text summary generation model is trained with the joint loss corresponding to the text summary automatic generation layer, the text category adaptation layer and the text style adaptation layer.

[0012] Optionally, during the text summary generation model training process:

[0013] The encoding layer encodes the training text to obtain an encoding feature vector of the training text;

[0014] The interactive layer decodes the encoded feature vector of the training text to obtain a decoded feature vector of the training text;

[0015] The text summary automatic generation layer is used to obtain a predicted summary of the training text based on the decoded feature vector of the training text, and the error between the predicted summary of the training text and the summary label annotated with the training text represents the loss corresponding to the text summary automatic generation layer;

[0016] The text category adaptation layer is used to obtain the predicted category of the training text based on the decoded feature vector of the training text, and the error between the predicted category of the training text and the category label annotated with the training text represents the loss corresponding to the text category adaptation layer;

[0017] The text style adaptation layer is used to obtain the predicted style of the training text based on the decoded feature vector of the training text, and the error between the predicted style of the training text and the style label annotated with the training text represents the loss corresponding to the text style adaptation layer.

[0018] Optionally, the training text is determined in the following manner:

[0019] Get data crawled by web crawlers;

[0020] Based on preset regular expression rules, the crawled data is cleaned to obtain the training text.

[0021] Optionally, determining the summary label, text category label and text style label of the training text based on the idea of ​​distant supervision includes:

[0022] The summary, text category, and text style of the training text are annotated by an external platform to determine the summary label, text category label, and text style label of the training text.

[0023] Optionally, the encoding layer in the text summary generation model is implemented using the embedding layer of the GPT-2 network, and the interaction layer in the text summary generation model is implemented using the interaction layer of the GPT-2 network; the text category adaptation layer and the text style adaptation layer in the text summary generation model are implemented using a self-attention mechanism.

[0024] A text summary generation model training device, the device comprising:

[0025] A text summary generation model acquisition unit, used to acquire a pre-built text summary generation model, wherein the text summary generation model includes an encoding layer, an interaction layer, a text summary automatic generation layer, a text category adaptation layer, and a text style adaptation layer;

[0026] A training text acquisition unit, used to acquire training text;

[0027] A label determination unit, used to determine the summary label, text category label and text style label of the training text based on the idea of ​​remote supervision;

[0028] The training unit is used to use the training text as a training sample, the summary label, text category label and text style label of the training text as sample labels, and train the text summary generation model with the joint loss corresponding to the text summary automatic generation layer, the text category adaptation layer and the text style adaptation layer.

[0029] Optionally, the label determination unit is specifically configured to determine the summary label, text category label and text style label of the training text by utilizing the summary, text category and text style annotated by an external platform on the training text.

[0030] Optionally, the encoding layer in the text summary generation model is implemented using the embedding layer of the GPT-2 network, the interaction layer in the text summary generation model is implemented using the interaction layer of the GPT-2 network, and the text category adaptation layer and the text style adaptation layer in the text summary generation model are implemented using a self-attention mechanism.

[0031] A text summary generation model training device, comprising a memory and a processor;

[0032] The memory is used to store programs;

[0033] The processor is used to execute the program to implement the various steps of the text summary generation model training method as described above.

[0034] A readable storage medium stores a computer program, which, when executed by a processor, implements the various steps of the text summary generation model training method as described above.

[0035] By means of the above technical scheme, the present application discloses a text summary generation model training method, related equipment and readable storage medium. Obtain a pre-built text summary generation model, the text summary generation model includes an encoding layer, an interaction layer, a text summary automatic generation layer, a text category adaptation layer and a text style adaptation layer; obtain training text; determine the summary label, text category label and text style label of the training text based on the idea of ​​far supervision; use the training text as a training sample, the summary label, text category label and text style label of the training text as sample labels, and train the text summary generation model with the joint loss corresponding to the text summary automatic generation layer, the text category adaptation layer and the text style adaptation layer. When the above method is used to train the text summary generation model, since the summary label, text category label and text style label of the training text are determined by the idea of ​​far supervision, there is no need to rely on manually annotated data. The text summary generation model composed of the encoding layer, the interaction layer, the text summary automatic generation layer, the text category adaptation layer and the text style adaptation layer can solve the polysemy and gradient disappearance problems of the sequence-to-sequence model based on the long short-term memory network. Moreover, the influence of text category and text style on summary content is considered during training. Therefore, using the text summary generation model obtained through the above training to automatically generate text summaries can improve the accuracy of the generated text summaries. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0037] Figure 1 A flowchart of a text summary generation model training method disclosed in an embodiment of the present application;

[0038] Figure 2 A structural diagram of a text summary generation model disclosed in an embodiment of the present application;

[0039] Figure 3A schematic diagram of the training and application of the text summary generation model disclosed in the embodiment of the present application;

[0040] Figure 4 This is a schematic diagram of the traditional GPT-2 network framework;

[0041] Figure 5 This is a schematic diagram of the structure of the GPT-2 network transformer decoder layer;

[0042] Figure 6 This is a structural example diagram of a text summary generation model disclosed in an embodiment of the present application;

[0043] Figure 7 A flowchart of a method for determining training text disclosed in an embodiment of the present application;

[0044] Figure 8 A schematic diagram of the structure of a text summary generation model training device disclosed in an embodiment of the present application;

[0045] Fig. 9 This is a hardware structure block diagram of a text summary generation model training device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0047] Next, the text summary generation model training method provided in this application is introduced through the following examples.

[0048] Reference Figure 1 , Figure 1 The following is a flow chart of a text summary generation model training method disclosed in an embodiment of the present application. The method may include:

[0049] Step S101: obtaining a pre-built text summary generation model, wherein the text summary generation model includes an encoding layer, an interaction layer, a text summary automatic generation layer, a text category adaptation layer, and a text style adaptation layer.

[0050] For easier understanding, please refer to Figure 2 , Figure 2This is a schematic diagram of the structure of a text summary generation model disclosed in an embodiment of the present application, which includes an encoding layer, an interaction layer, a text summary automatic generation layer, a text category adaptation layer, and a text style adaptation layer. Each module can be implemented using a pre-trained neural network structure, which will be described in detail in the following embodiments.

[0051] Step S102: Obtain training text.

[0052] In this application, the training text can be any text

[0053] Step S103: Based on the idea of ​​remote supervision, the summary label, text category label and text style label of the training text are determined.

[0054] Taking into account the manual annotation of data, this is a very time-consuming and laborious process, especially for text summarization tasks. Each time a annotation result is obtained, a large number of article paragraphs need to be read. At the same time, different people have different summary generation standards and summarization methods, which will eventually cause different output result distributions, thus affecting the experimental results. In order to solve the above problems, in this application, the summary label of the training text can be determined based on the idea of ​​remote supervision. As an implementable method, an external platform can be used to annotate the summary, text category, and text style of the training text to determine the summary label, text category label, and text style label of the training text.

[0055] In this application, text category is used to characterize the category to which the text belongs, such as literature, science, etc., and text style is used to characterize the text style of the text, such as funny style, news release style, etc. The text category labels and text style labels of the training text annotations can be manually annotated.

[0056] Step S104: using the training text as a training sample, using the summary label, text category label and text style label of the training text as sample labels, and training the text summary generation model with the joint loss corresponding to the text summary automatic generation layer, the text category adaptation layer and the text style adaptation layer.

[0057] It should be noted that once the performance of the text summary generation model is satisfied, the text summary generation model can be used to perform the text summary generation task, that is, input text to the text summary generation model, and the text summary generation model generates a summary. If the performance of the text summary generation model trained with the crawled data is not satisfactory, you can crawl data again, clean the data, use the remote supervision idea to get the label, and continue to train the text summary generation model until the performance of the text summary generation model is satisfied. For details, please refer to Figure 3 As shown, Figure 3A schematic diagram of the training and application of the text summary generation model disclosed in the embodiment of the present application.

[0058] This embodiment discloses a method for training a text summary generation model. A pre-built text summary generation model is obtained, and the text summary generation model includes an encoding layer, an interaction layer, a text summary automatic generation layer, a text category adaptation layer, and a text style adaptation layer; training text is obtained; based on the idea of ​​far supervision, the summary label, text category label, and text style label of the training text are determined; the training text is used as a training sample, the summary label, text category label, and text style label of the training text are used as sample labels, and the text summary generation model is trained with the joint loss corresponding to the text summary automatic generation layer, the text category adaptation layer, and the text style adaptation layer. When the above method is used to train the text summary generation model, since the summary label, text category label, and text style label of the training text are determined by the idea of ​​far supervision, there is no need to rely on manually annotated data. The text summary generation model composed of the encoding layer, the interaction layer, the text summary automatic generation layer, the text category adaptation layer, and the text style adaptation layer can solve the polysemy and gradient disappearance problems of the sequence-to-sequence model based on the long short-term memory network. Moreover, the influence of text category and text style on the summary content is considered during training. Therefore, using the text summary generation model obtained through the above training to automatically generate text summaries can improve the accuracy of the generated text summaries.

[0059] In another embodiment of the present application, the functional implementation of each module in the text summary generation model training process is introduced as follows:

[0060] The encoding layer encodes the training text to obtain an encoding feature vector of the training text;

[0061] The interactive layer decodes the encoded feature vector of the training text to obtain a decoded feature vector of the training text;

[0062] The text summary automatic generation layer is used to obtain a predicted summary of the training text based on the decoded feature vector of the training text, and the error between the predicted summary of the training text and the summary label annotated with the training text represents the loss corresponding to the text summary automatic generation layer;

[0063] The text category adaptation layer is used to obtain the predicted category of the training text based on the decoded feature vector of the training text, and the error between the predicted category of the training text and the category label annotated with the training text represents the loss corresponding to the text category adaptation layer;

[0064] The text style adaptation layer is used to obtain the predicted style of the training text based on the decoded feature vector of the training text, and the error between the predicted style of the training text and the style label annotated with the training text represents the loss corresponding to the text style adaptation layer.

[0065] As an implementable embodiment, the encoding layer in the text summary generation model is implemented by the embedding layer (embedding layer) of the GPT-2 network, and the interaction layer in the text summary generation model is implemented by the interaction layer (transformer decoder layer) of the GPT-2 network.

[0066] For ease of understanding, the GPT-2 network is introduced in this application as follows, referring to Figure 4 , Figure 4 This is a schematic diagram of the traditional GPT-2 network framework. Figure 4 As shown in the figure, the GPT-2 network is mainly composed of an embedding layer and an interaction layer. The embedding layer includes Token Embedding and Position Embedding, and the transformer decoder layer includes multiple stacked Autoregressive Transformer Decoder Blocks. The input of the GPT-2 network is the different tokens obtained after the text is segmented. After the different tokens pass through the embedding layer, they can get the input of the transformer decoder layer, and finally can be decoded by the transformer decoder layer to get the downstream output.

[0067] For both Token Embedding and Position Embedding, different indexes are embedded based on an embedding matrix. Taking Token embedding as an example, let the vocabulary be V and the embedding dimension be d model , then the embedding matrix has a size of |V|×d model Tensor, different tokens have different index values ​​in the vocabulary. By indexing the embedding matrix row by row according to the index value, the embedding tensor of the token can be obtained, as shown in the following formula:

[0068]

[0069] Among them, e i is token i The embedding representation of , j is a token i The index in V, M is the embedding matrix, and M(j) represents the j-th row element of M.

[0070] Reference Figure 5 , Figure 5 This is a schematic diagram of the structure of the GPT-2 network transformer decoder layer. Figure 5 As shown in the figure, the transformer decoder layer is composed of a masked multi-head self-attention network layer, an LN normalization layer (Layer Norm) and a feed forward network layer (Feed Forward), wherein the masked multi-head self-attention network layer mainly processes the text based on the self-attention mechanism.

[0071] The attention mechanism is a mechanism that re-weights and updates input information based on its context information. Generally, query, key and value are set in the attention. The input information is used as the query, the context information is used as the key and value, and the query is used to query the key to obtain the relevance, and the update is obtained from the value based on the relevance. In the self-attention mechanism, the query, key and value are all input information x, as shown in the following formula, and the relevance is obtained as follows:

[0072]

[0073] Among them, Q, K and V represent the query, key and value in the attention mechanism respectively, A is the correlation matrix corresponding to the query key, L is the length of the input text, α ij is the element in the i-th row and j-th column of matrix A, w ij is α normalized by the softmax function ij , together form the weight matrix W.

[0074] According to human reading habits, the model should not know the information of the back of the text when processing the front of the text. Therefore, in the attention, the transformer decoder layer uses a mask mechanism to ensure that the information of the back of the text is not visible to the front when the entire text is input together. The mask mechanism is as follows:

[0075]

[0076] Overall, the input x is given to the transformer decoder layer, which performs the following processing in the following order:

[0077] x=x+Attention(Q:LN(x),K:LN(x),V:LN(x))

[0078] x=x+GeLU(W·LN(x)+b)

[0079] Among them, LN stands for layer normalization.

[0080] As an implementable embodiment, the text category adaptation layer and the text style adaptation layer in the text summary generation model are implemented using a self-attention mechanism.

[0081] Reference Figure 6 , Figure 6 This is a structural example diagram of a text summary generation model disclosed in an embodiment of the present application. In this example, the encoding layer in the text summary generation model is implemented by the embedding layer of the GPT-2 network, the interaction layer is implemented by the interaction layer of the GPT-2 network, and the text category adaptation layer and the text style adaptation layer are implemented by the self-attention mechanism.

[0082] It should be noted that the loss corresponding to the automatic generation layer of text summarization is Loss generate , the loss corresponding to the text category adaptation layer is Loss class , the loss corresponding to the text style adaptation layer style , the joint loss Loss is specifically Loss = Loss generate +αLoss class +βLoss style , where α and β represent the adjustment values ​​of the loss corresponding to the text category adaptation layer and the loss corresponding to the text style adaptation layer, respectively. The loss corresponding to the text category adaptation layer and the loss corresponding to the text style adaptation layer can use the cross entropy loss, and the loss corresponding to the text summary automatic generation layer can be the maximum likelihood probability, which can be specifically expressed by the following formula:

[0083]

[0084] Where y represents the true label of the corresponding task, and p represents the predicted probability of the corresponding task.

[0085] As an implementable method, the text summary automatic generation layer in the text summary generation model often uses softmax to normalize the output words when selecting words, and obtains sentence output based on beam search. The generated summary can be obtained by combining the sentences output at each moment and outputting multiple sentences that the model considers to be the most reliable (the number is determined by BeamWidth).

[0086] Beam Search means calculating the probability of all output words at the current level and sorting them in ascending order, but only retaining a certain number of possible results (the number is determined by the Beam Width), and then expanding based on these possible results, iterating the above actions until the search ends and returning the best solution (the one with the highest probability).

[0087] In another embodiment of the present application, the method for determining the training text is described, referring to Figure 7 , Figure 7 This is a flow chart of a method for determining a training text disclosed in an embodiment of the present application. The method may include:

[0088] Step S201: Acquire data crawled by a web crawler.

[0089] In this application, web crawler technology can be used to crawl relevant data according to project requirements. A web crawler is a program or script that automatically crawls World Wide Web information according to certain rules. First, an http request is made to obtain the response of the target server, and then the returned response is parsed using relevant scripts to obtain important resources.

[0090] Step S202: Based on preset regular expression rules, the crawled data is cleaned to obtain the training text.

[0091] Since the data crawled from the web often contains garbled characters, it is necessary to clean the crawled data. Data cleaning refers to the process of re-examining and verifying the data, with the aim of deleting duplicate information, correcting existing errors, and providing data consistency.

[0092] Using relevant regular expression rules to clean the crawled data can make the final output text as fluent as possible and people can understand the meaning of the text by reading it.

[0093] For ease of understanding, for example, when there are a large number of irrelevant URLs in the original text, regular expression patterns such as re.(https?|ftp|file): / / [-A-Za-z0-9+&@# / %?=~_|!:,.;]+[-A-Za-z0-9+&@# / %=~_|] can be used to match the URLs and then delete them.

[0094] The text summary generation model training device disclosed in the embodiment of the present application is described below. The text summary generation model training device described below and the text summary generation model training method described above can refer to each other.

[0095] Reference Figure 8 , Figure 8 This is a schematic diagram of the structure of a text summary generation model training device disclosed in an embodiment of the present application. Figure 8 As shown, the text summary generation model training device may include:

[0096] A text summary generation model acquisition unit 11 is used to acquire a pre-built text summary generation model, wherein the text summary generation model includes an encoding layer, an interaction layer, a text summary automatic generation layer, a text category adaptation layer, and a text style adaptation layer;

[0097] A training text acquisition unit 12, used to acquire training text;

[0098] A label determination unit 13, used to determine the summary label, text category label and text style label of the training text based on the remote supervision idea;

[0099] The training unit 14 is used to use the training text as a training sample, the summary label, text category label and text style label of the training text as sample labels, and train the text summary generation model with the joint loss corresponding to the text summary automatic generation layer, the text category adaptation layer and the text style adaptation layer.

[0100] As an implementable embodiment, the label determination unit is specifically used to determine the summary label, text category label and text style label of the training text by utilizing the summary, text category and text style annotated by the external platform on the training text.

[0101] As an implementable embodiment, the encoding layer in the text summary generation model is implemented by the embedding layer of the GPT-2 network, the interaction layer in the text summary generation model is implemented by the interaction layer of the GPT-2 network, and the text category adaptation layer and the text style adaptation layer in the text summary generation model are implemented by a self-attention mechanism.

[0102] Reference Fig. 9 , Fig. 9 The hardware structure block diagram of the text summary generation model training device provided in the embodiment of the present application is shown in FIG. Fig. 9 ,The hardware structure of the text summary generation model training device may include: at least one processor 1, at least one communication interface 2, at least one memory 3 and at least one communication bus 4;

[0103] In the embodiment of the present application, the number of the processor 1, the communication interface 2, the memory 3, and the communication bus 4 is at least one, and the processor 1, the communication interface 2, and the memory 3 communicate with each other through the communication bus 4;

[0104] The processor 1 may be a central processing unit CPU, or an application-specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention, etc.;

[0105] The memory 3 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory;

[0106] The memory stores a program, and the processor can call the program stored in the memory, wherein the program is used to:

[0107] Obtaining a pre-built text summary generation model, wherein the text summary generation model includes an encoding layer, an interaction layer, a text summary automatic generation layer, a text category adaptation layer, and a text style adaptation layer;

[0108] Get training text;

[0109] Based on the idea of ​​distant supervision, determining the summary label, text category label and text style label of the training text;

[0110] The training text is used as a training sample, the summary label, text category label and text style label of the training text are used as sample labels, and the text summary generation model is trained with the joint loss corresponding to the text summary automatic generation layer, the text category adaptation layer and the text style adaptation layer.

[0111] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0112] The embodiment of the present application further provides a readable storage medium, which may store a program suitable for execution by a processor, wherein the program is used to:

[0113] Obtaining a pre-built text summary generation model, wherein the text summary generation model includes an encoding layer, an interaction layer, a text summary automatic generation layer, a text category adaptation layer, and a text style adaptation layer;

[0114] Get training text;

[0115] Based on the idea of ​​distant supervision, determining the summary label, text category label and text style label of the training text;

[0116] The training text is used as a training sample, the summary label, text category label and text style label of the training text are used as sample labels, and the text summary generation model is trained with the joint loss corresponding to the text summary automatic generation layer, the text category adaptation layer and the text style adaptation layer.

[0117] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0118] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0119] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0120] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A text summary generation model training method, characterized in that: The method comprises: Obtaining a pre-built text summary generation model, wherein the text summary generation model includes an encoding layer, an interaction layer, a text summary automatic generation layer, a text category adaptation layer, and a text style adaptation layer; Get training text; Based on the idea of ​​distant supervision, determining the summary label, text category label and text style label of the training text; The training text is used as a training sample, the summary label, text category label and text style label of the training text are used as sample labels, and the text summary generation model is trained with the joint loss corresponding to the text summary automatic generation layer, the text category adaptation layer and the text style adaptation layer; During the text summarization model training process: The encoding layer encodes the training text to obtain an encoding feature vector of the training text; The interactive layer decodes the encoded feature vector of the training text to obtain a decoded feature vector of the training text; The text summary automatic generation layer is used to obtain a predicted summary of the training text based on the decoded feature vector of the training text, and the error between the predicted summary of the training text and the summary label annotated with the training text represents the loss corresponding to the text summary automatic generation layer; The text category adaptation layer is used to obtain the predicted category of the training text based on the decoded feature vector of the training text, and the error between the predicted category of the training text and the category label annotated with the training text represents the loss corresponding to the text category adaptation layer; The text style adaptation layer is used to obtain the predicted style of the training text based on the decoded feature vector of the training text, and the error between the predicted style of the training text and the style label annotated with the training text represents the loss corresponding to the text style adaptation layer; The training text is determined in the following manner: Get data crawled by web crawlers; Based on a preset regular expression rule, the crawled data is cleaned to obtain the training text; The method of determining the summary label, text category label and text style label of the training text based on the idea of ​​distant supervision includes: Determine the summary label, text category label and text style label of the training text by using the summary, text category and text style annotated by the external platform; The encoding layer in the text summary generation model is implemented by the embedding layer of the GPT-2 network, the interaction layer in the text summary generation model is implemented by the interaction layer of the GPT-2 network, and the text category adaptation layer and the text style adaptation layer in the text summary generation model are implemented by the self-attention mechanism.

2. A text summary generation model training device, characterized in that: For executing the method according to claim 1, the device comprises: A text summary generation model acquisition unit, used to acquire a pre-built text summary generation model, wherein the text summary generation model includes an encoding layer, an interaction layer, a text summary automatic generation layer, a text category adaptation layer, and a text style adaptation layer; A training text acquisition unit, used to acquire training text; A label determination unit, used to determine a summary label, a text category label and a text style label of the training text based on a remote supervision concept; The training unit is used to use the training text as a training sample, the summary label, text category label and text style label of the training text as sample labels, and train the text summary generation model with the joint loss corresponding to the text summary automatic generation layer, the text category adaptation layer and the text style adaptation layer.

3. The device according to claim 2, characterized in that The label determination unit is specifically used to determine the summary label, text category label and text style label of the training text by using the summary, text category and text style annotated by the external platform on the training text.

4. A text summary generation model training device, characterized in that: including memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the text summary generation model training method as described in claim 1.

5. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the text summary generation model training method as described in claim 1 is implemented.

Images