Text abstract generation method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a text summary generation method and device, electronic equipment and storage medium. The method comprises: obtaining a target text and a summary generation prompt text; inputting the target text and the summary generation prompt text into a target neural network model to obtain a text summary; the training process of the target neural network model comprises: obtaining training sample data; inputting sample text data into a preset neural network model to obtain a first prediction probability sequence corresponding to a first candidate word contained in a first summary label, calculating a first loss based on the first summary label and the first prediction probability sequence; obtaining a first reference probability sequence of the first candidate word, calculating a second loss based on the first prediction probability sequence of the first candidate word and the first reference probability sequence; updating the parameters of the preset neural network model according to the first loss and the second loss, and determining the target neural network model based on the updated preset neural network model. The present embodiment can improve the accuracy of text summary generation.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, and in particular, to a text summary generation method and device, electronic equipment and storage medium. BACKGROUND

[0002] With the development of the film and television industry, the number of film and television scripts is also increasing. Since the amount of text contained in the film and television script is relatively large, and it includes relatively complex story plots and character relationships, it is very difficult to quickly understand the content of the film and television script and evaluate whether the script is suitable for filming. Therefore, it is necessary to generate an accurate script summary based on the film and television script.

[0003] In related technologies, the film and television script is often input into a preset neural network model, and the neural network model can extract the required content from the script to generate a text summary, so that the object can quickly understand the script content, greatly improving the reading efficiency of the film and television script.

[0004] However, the accuracy of text summary generation for film and television scripts in related technologies is low. SUMMARY

[0005] The embodiments of the present disclosure provide a text summary generation method, device, electronic equipment and storage medium, which can improve the accuracy of summary generation for text content.

[0006] According to an aspect of the present disclosure, a text summary generation method is provided, comprising:

[0007] obtaining a target text and a summary generation prompt text;

[0008] inputting the target text and the summary generation prompt text into a pre-trained target neural network model to obtain a text summary corresponding to the target text;

[0009] The training process of the target neural network model comprises:

[0010] obtaining training sample data, the training sample data comprising sample text data and a first summary label corresponding to the sample text data;

[0011] inputting the sample text data into a preset neural network model to obtain at least one first prediction probability sequence corresponding to at least one first candidate word contained in the first summary label output by the preset neural network model, the first prediction probability sequence comprising output probabilities corresponding to a plurality of reference words in a reference word library, and calculating a first loss based on the first summary label and the at least one first prediction probability sequence;

[0012] obtain a first reference probability sequence corresponding to each of the first candidate words, and calculate a second loss based on the first prediction probability sequence corresponding to each of the first candidate words and the first reference probability sequence, a first uniformity corresponding to a first probability distribution of the first reference probability sequence being higher than a second uniformity corresponding to a second probability distribution of the first prediction probability sequence;

[0013] calculate a first target loss according to the first loss and the second loss, update parameters of the preset neural network model based on the first target loss, and determine a target neural network model based on the updated preset neural network model.

[0014] According to an aspect of the present disclosure, a text summary generation apparatus is provided, comprising:

[0015] an obtaining unit configured to obtain a target text and a summary generation prompt text;

[0016] a generating unit configured to input the target text and the summary generation prompt text into a pre-trained target neural network model to obtain a text summary corresponding to the target text;

[0017] The training process of the target neural network model comprises:

[0018] obtain training sample data, the training sample data comprising sample text data and a first summary label corresponding to the sample text data;

[0019] input the sample text data into a preset neural network model to obtain at least one first prediction probability sequence corresponding to at least one first candidate word contained in the first summary label and output by the preset neural network model, the first prediction probability sequence comprising output probabilities corresponding to a plurality of reference words in a reference word library, and calculate a first loss based on the first summary label and the at least one first prediction probability sequence;

[0020] obtain a first reference probability sequence corresponding to each of the first candidate words, and calculate a second loss based on the first prediction probability sequence corresponding to each of the first candidate words and the first reference probability sequence, a first uniformity corresponding to a first probability distribution of the first reference probability sequence being higher than a second uniformity corresponding to a second probability distribution of the first prediction probability sequence;

[0021] calculate a first target loss according to the first loss and the second loss, update parameters of the preset neural network model based on the first target loss, and determine a target neural network model based on the updated preset neural network model.

[0022] Optionally, in an embodiment, the calculating the second loss based on the first predicted probability sequence corresponding to each of the first candidate words and the first reference probability sequence comprises:

[0023] calculating a first difference term corresponding to each of the first candidate words based on the first predicted probability sequence corresponding to each of the first candidate words and the first reference probability sequence;

[0024] calculating the second loss based on the first difference term.

[0025] Optionally, in an embodiment, the first summary label comprises at least two first candidate words, and before the calculating the second loss based on the first difference term, the method further comprises:

[0026] determining at least one candidate word sequence based on the at least two first candidate words comprised in the first summary label;

[0027] calculating a second difference term corresponding to each of the candidate word sequence based on the first predicted probability sequence corresponding to each of the first candidate words in each of the candidate word sequence and the corresponding first reference probability sequence;

[0028] the calculating the second loss based on the first difference term comprises:

[0029] calculating the second loss based on the first difference term and the second difference term.

[0030] Optionally, in an embodiment, the calculating the second difference term corresponding to each of the candidate word sequence based on the first predicted probability sequence corresponding to each of the first candidate words in each of the candidate word sequence and the corresponding first reference probability sequence comprises:

[0031] generating a sequence predicted probability distribution based on the first predicted probability sequence corresponding to each of the first candidate words in each of the candidate word sequence, and generating a sequence reference probability distribution based on the first reference probability sequence corresponding to each of the first candidate words in the candidate word sequence;

[0032] calculating the second difference term corresponding to each of the candidate word sequence based on the sequence predicted probability distribution corresponding to each of the candidate word sequence and the sequence reference probability distribution.

[0033] Optionally, in an embodiment, the generating the sequence predicted probability distribution based on the first predicted probability sequence corresponding to each of the first candidate words in each of the candidate word sequence comprises:

[0034] The first prediction probability sequence corresponding to each first candidate word in each candidate word sequence is spliced according to a first candidate word arrangement order to obtain a spliced sequence;

[0035] A sequence prediction probability distribution is generated based on the spliced sequence and a number of first candidate words contained in the candidate word sequence.

[0036] Optionally, in an embodiment, the determining of the target neural network model based on the updated preset neural network model comprises:

[0037] A second summary label corresponding to the sample text data is obtained, and a first quality score of the first summary label is higher than a second quality score of the second summary label;

[0038] The sample text data is input into the updated preset neural network model to obtain at least one second prediction probability sequence corresponding to the at least one first candidate word contained in the first summary label and output by the preset neural network model, and at least one third prediction probability sequence corresponding to at least one second candidate word contained in the second summary label;

[0039] A third loss is calculated based on the first summary label, the second summary label, the second prediction probability sequence, and the third prediction probability sequence;

[0040] A second reference probability sequence corresponding to each first candidate word is obtained, and a fourth loss is calculated based on the second prediction probability sequence corresponding to each first candidate word and the second reference probability sequence, and the third prediction probability sequence corresponding to each second candidate word;

[0041] A second target loss is calculated according to the third loss and the fourth loss, and parameters of the preset neural network model are updated based on the second target loss to determine a target neural network model.

[0042] Optionally, in an embodiment, the calculating of the fourth loss based on the second prediction probability sequence corresponding to each first candidate word and the second reference probability sequence, and the third prediction probability sequence corresponding to each second candidate word comprises:

[0043] A first candidate word difference term corresponding to the first summary label is calculated based on the second prediction probability sequence corresponding to each first candidate word and the second reference probability sequence;

[0044] A second candidate word difference term corresponding to the second summary label is calculated based on the third prediction probability sequence corresponding to each second candidate word;

[0045] The fourth loss is calculated according to the first candidate word difference item and the second candidate word difference item.

[0046] Optionally, in an embodiment, the fourth loss is calculated according to the first candidate word difference item and the second candidate word difference item, comprising:

[0047] A total difference item is determined based on the first candidate word difference item and the second candidate word difference item.

[0048] The total difference item is normalized to obtain the fourth loss.

[0049] Optionally, in an embodiment, the total difference item is determined based on the first candidate word difference item and the second candidate word difference item, comprising:

[0050] A first adjustment parameter corresponding to the first candidate word difference item and a second adjustment parameter corresponding to the second candidate word difference item are obtained.

[0051] A total difference item is calculated based on the first adjustment parameter, the second adjustment parameter, the first candidate word difference item and the second candidate word difference item.

[0052] Optionally, in an embodiment, the first target loss is calculated according to the first loss and the second loss, comprising:

[0053] A first loss weight corresponding to the first loss is obtained, and a second loss weight corresponding to the second loss is obtained.

[0054] The first loss and the second loss are weighted calculated based on the first loss weight and the second loss weight, to obtain the first target loss.

[0055] Optionally, in an embodiment, the second loss weight corresponding to the second loss is obtained, comprising:

[0056] A weight curve corresponding to the second loss is obtained.

[0057] A second loss weight is determined from the weight curve based on an iteration round in which a parameter of the preset neural network model is updated in a training period.

[0058] In the text abstract generation method of the embodiments of the present disclosure, the target text and the abstract generation prompt text are obtained; the target text and the abstract generation prompt text are input into a pre-trained target neural network model to obtain a text abstract corresponding to the target text; the training process of the target neural network model includes: obtaining training sample data, the training sample data including sample text data and a first abstract label corresponding to the sample text data; inputting the sample text data into a preset neural network model to obtain at least one first prediction probability sequence corresponding to at least one first candidate word contained in the first abstract label output by the preset neural network model, the first prediction probability sequence including output probabilities corresponding to a plurality of reference words in a reference word library, and calculating a first loss based on the first abstract label and the at least one first prediction probability sequence; obtaining a first reference probability sequence corresponding to each first candidate word, and calculating a second loss based on the first prediction probability sequence corresponding to each first candidate word and the first reference probability sequence, the first probability distribution of the first reference probability sequence corresponding to a first balance higher than a second balance corresponding to a second probability distribution of the first prediction probability sequence; calculating a first target loss according to the first loss and the second loss, and updating parameters of the preset neural network model based on the first target loss, and determining the target neural network model based on the updated preset neural network model.

[0059] In this way, when training the target neural network model for generating a text abstract, the embodiments of the present disclosure guide the prediction probability sequence predicted by the target neural network model at each word position by setting the reference probability sequence. Since the probability distribution balance of the reference probability sequence is higher than the probability distribution balance of the prediction probability sequence, the overfitting problem caused by the prediction output of the target neural network model at each word position being too close to the label can be avoided. In this way, the accuracy of predicting the word at each word position in the text abstract generation process can be improved, and the accuracy of generating a text abstract can be further improved.

[0060] Other features and advantages of the present disclosure will be set forth in the description that follows, and in part will become apparent to those skilled in the art upon examination of the description, or can be learned by practice of the present disclosure. The objects and other advantages of the present disclosure can be realized and attained by the structure particularly pointed out in the description, claims, and drawings. BRIEF DESCRIPTION OF DRAWINGS

[0061] The accompanying drawings are included to provide a further understanding of the technical solutions of the present disclosure, and constitute a part of the specification, and are used to explain the technical solutions of the present disclosure together with the embodiments of the present disclosure, and do not constitute a limitation on the technical solutions of the present disclosure.

[0062] Figure 1 is a framework diagram of a system to which the text abstract generation method according to the embodiments of the present disclosure is applied;

[0063] Figure 2A is a schematic diagram of an embodiment of the present disclosure applied in a script plot summary generation scenario;

[0064] Figure 2B is another schematic diagram of an embodiment of the present disclosure applied in a script plot summary generation scenario;

[0065] Figure 2C is another schematic diagram of an embodiment of the present disclosure applied in a script plot summary generation scenario;

[0066] Figure 3 is a flowchart of a text summary generation method according to an embodiment of the present disclosure;

[0067] Figure 4 is a schematic diagram of inputting text in a text summary generation interface according to an embodiment of the present disclosure;

[0068] Figure 5 is a structural schematic diagram of a target neural network model according to an embodiment of the present disclosure;

[0069] Figure 6 is a flowchart of a target neural network model training process according to an embodiment of the present disclosure;

[0070] Figure 7 is a schematic diagram of a prediction probability sequence corresponding to a plurality of candidate words output by a preset neural network model according to an embodiment of the present disclosure;

[0071] Figure 8 is a schematic diagram of a weight curve corresponding to a second loss weight according to an embodiment of the present disclosure;

[0072] Figure 9 is a schematic diagram of a corresponding relationship between a first loss weight and a second loss weight according to an embodiment of the present disclosure;

[0073] Figure 10 is another flowchart of a text summary generation method according to an embodiment of the present disclosure;

[0074] Figure 11 is a schematic diagram of training a first target neural network model based on a reference neural network model according to an embodiment of the present disclosure;

[0075] Figure 12 is a structural schematic diagram of a text summary generation apparatus according to an embodiment of the present disclosure;

[0076] Figure 13 is a terminal structure diagram for implementing methods according to an embodiment of the present disclosure;

[0077] Figure 14 is a server structure diagram for implementing the methods according to one embodiment of the present disclosure. DETAILED DESCRIPTION

[0078] In order to make the purposes, technical solutions and advantages of the present disclosure clearer, the present disclosure will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present disclosure and do not limit the present disclosure.

[0079] Before the present disclosure is further described in detail, the terms and phrases involved in the embodiments of the present disclosure are explained, and the terms and phrases involved in the embodiments of the present disclosure are applicable to the following explanations:

[0080] Large Language Model (LLM): refers to a deep learning model trained using a large amount of text data, which can generate natural language text or understand the meaning of language text. LLM can handle a variety of natural language tasks such as text classification, question answering, dialogue, etc. The core idea of LLM is to learn the patterns and language structures of natural language through large-scale unsupervised training, store a large amount of natural language knowledge, and better understand and generate natural text.

[0081] Neural Network: Neural Networks is an important technology in machine learning, which is inspired by the structure and function of human brain. Neural networks are composed of a large number of neuron nodes, which interact through different connections (weights) to form a complex network. Neural networks can automatically learn the features in the data and handle complex problems through multi-level network structure.

[0082] In the related art, a large language model can be used to generate a summary of text content. However, for text content in a certain field, the large language model has not been trained specifically for this field, so the accuracy of summary generation is low. To this end, a pre-set sample text and corresponding summary label can be used to fine-tune the model, so that the model can learn the text content in a specific field, thereby improving the accuracy of text summary generation. However, although fine-tuning with labels can improve the accuracy of summary generation in a certain field, due to the limited number of labels, supervised training can cause the output of the model to approach the words in the limited labels, thereby causing the model to overfit during fine-tuning training. That is, when the model generates a summary corresponding to the text content, the generation probability of the words contained in the summary label is high, and the generation probability of other words that can more accurately express the text content is low. The similarity of the summaries generated for different text contents is high, and the precise word selection for summary generation of different text contents cannot be achieved, thereby resulting in low accuracy of text summary generation. To this end, the embodiments of the present disclosure provide a text summary generation method to improve the accuracy of text summary generation.

[0083] System architecture and scenario to which the embodiments of the present disclosure are applied

[0084] Figure 1 is a system architecture diagram to which a text summary generation method according to an embodiment of the present disclosure is applied. It includes a terminal 140, an Internet 130, a gateway 120, a server 110, and the like.

[0085] The terminal 140 includes desktop computers, laptop computers, PDAs (personal digital assistants), mobile phones, vehicle-mounted terminals, smart home appliances, special-purpose terminals, and the like. In addition, it can be a single device or a collection of multiple devices. For example, multiple devices are connected through a local area network, share a display device for collaborative work, and collectively constitute a terminal 140. The terminal 140 can communicate with the Internet 130 in a wired or wireless manner and exchange data.

[0086] The server 110 refers to a computer system that can provide certain services to the terminal 140. Compared with ordinary terminals 140, the server 110 has high requirements in terms of stability, security, performance, and the like. The server 110 can be a high-performance computer in a network platform, a cluster of multiple high-performance computers, a portion (e.g., a machine) of a high-performance computer, a combination of portions (e.g., machines) of multiple high-performance computers, and the like.

[0087] Gateway 120, also known as an internetwork connector or protocol converter, is a computer system or device that acts as a translator between two systems using different communication protocols, data formats, languages, or even completely different architectures. It enables network interconnection at the transport layer. Gateways can also provide filtering and security functions. Messages sent from terminal 140 to server 110 are forwarded to the corresponding server 110 through gateway 120. Messages sent from server 110 to terminal 140 are also forwarded to the corresponding terminal 140 through gateway 120.

[0088] The text summarization method of this disclosure can be executed by an electronic device, which can be a terminal 140 or a server 110. That is, the text summarization method can be executed by the terminal 140 or the server 110, or it can be executed by both the terminal 140 and the server 110.

[0089] This disclosure can be applied to various scenarios, such as generating summaries of complex script content. Specifically, this disclosure can be applied to various scenarios related to script content, such as... Figure 2A The scene shown summarizes the content of different scenes in the script; Figure 2B The scene shown outlines the development of the script's plot; and Figure 2C The scene shown illustrates the process of filtering through a large number of scripts.

[0090] The script may involve multiple characters, complex plots, timelines, and locations. It may also include multiple scenes, which may be related. Summarizing the content of different scenes based on specific criteria facilitates rapid organization of the scene content. For example... Figure 2A As shown, script 211 can be divided into multiple scenes 212; the text summary generation method of this embodiment generates a summary for each scene, resulting in a corresponding scene summary 213; organizing multiple scene summaries 213 yields a multi-scene summary 214. When it is desired to summarize the plots corresponding to characters A1 and A2 from the content of multiple scenes, the character plots can be summarized based on the multi-scene summary 214, resulting in the plots of character A1 215 and character A2 216.

[0091] For example Figure 2B As shown, script 221 can be broken down into multiple scenes 222. After generating scene summaries 223 for each scene, the entire script can be analyzed based on these summaries to obtain plot development 224. Plot development 224 can help users quickly understand the script.

[0092] For example Figure 2CAs shown, for the script 231, it can be split into multiple scenes 232; the text summarization method of the present disclosure example can generate a summary for the content of each scene to obtain a scene summary 233, and can also generate a role summary for different roles in each scene based on the content of each scene to obtain a role summary 234. Since the summary content has summarization ability and is relatively concise, the scene summary 233 can quickly understand the content of each scene, and the target scene 235 can be obtained by screening the scene content, so as to retain important target scenes and discard unimportant scenes; the role summary 234 can quickly understand the content of each role, and the target role 236 can be obtained by screening the role, so as to retain important roles for shooting and discard irrelevant roles.

[0093] As can be seen, in the face of relatively complex script content, the text summary generation method of the present disclosure embodiment can generate an accurate summary to quickly understand the text content based on the summary.

[0094] Overall description of the present disclosure embodiment

[0095] According to one embodiment of the present disclosure, a text summary generation method is provided. The text summary generation method provided by the present disclosure embodiment can be applied to scenes such as script content summary generation. As shown in the following Figure 3 As shown, one flowchart of the text summary generation method provided by the present disclosure. The text summary generation method can include:

[0096] Step 310, obtaining a target text and a summary generation prompt text.

[0097] The target text can be a text that needs to be summarized. The target text can be a text content input by an object through a text input box, or a content uploaded in the form of a file. Obtaining the target text can be by obtaining the text content input by the object in the text input box as the target text, or by parsing the text file uploaded by the object to obtain the target text.

[0098] The summary generation prompt text can be a text that provides guidance for the summary generation task of the model. The summary generation prompt text can contain the requirements of the text summary generation, so that the model generates a text summary according to the summary generation requirements. For example, in a script summary generation task, the summary generation prompt text is: "Give a plot summary based on the given script text, and require to describe the events, time, place, cause, process and result in the script, within 80 words." Through this summary generation prompt text, the model can generate a plot summary based on the script text, and the model is required to generate a summary that contains the specified content, and the summary word limit is specified.

[0099] The summary generation prompt text can be pre-set based on the summary generation scenario. Different summary generation prompt texts can be pre-set for different summary generation scenarios. For example, corresponding summary generation prompt texts can be pre-set for script summary generation tasks, and corresponding summary generation prompt texts can also be pre-set for news summary generation tasks. After obtaining the target text, the corresponding summary generation prompt text can be automatically selected based on the type of the target text, or the subject can select the summary generation prompt text for performing the summary generation task from the multiple pre-set summary generation prompt texts according to the summary generation requirement.

[0100] The summary generation prompt text can also be a prompt text input by the subject in the text input box. The prompt text input by the subject in the text input box is obtained as the summary generation prompt text. The text input by the subject through the text input box can include the summary generation prompt text and the target text, for example Figure 4 As shown in FIG. 4, the subject inputs “give a plot summary based on the given script text, and require to describe the events, time, place, cause, process and result in the script, within 80 words. Script text: XXX.” in the text input box 411 in the text summary generation interface 410. The input text is uploaded to the summary generation control 412 to trigger the summary generation. In the text input by the subject, “give a plot summary based on the given script text, and require to describe the events, time, place, cause, process and result in the script, within 80 words” is the summary generation prompt text, and “XXX” is the target text. After obtaining the text input by the subject through the text summary generation interface 410, the text can be recognized to parse the summary generation prompt text and the target text therefrom.

[0101] Step 320, input the target text and the summary generation prompt text into the pre-trained target neural network model to obtain a text summary corresponding to the target text.

[0102] The target neural network model is a neural network model for generating a summary based on the summary generation prompt text and the target text. The target neural network model can be a large language model with language recognition and generation capabilities.

[0103] In an embodiment, the model structure of the target neural network model can be as shown in FIG. 5. Figure 5As shown, the target neural network model 500 includes a text encoding layer 510, a position encoding layer 520, an encoding network 530, and a linear transformation layer 540. The text encoding layer 510 can be used to encode the input text into a fixed-length context vector that captures the contextual semantic information in the input text. The position encoding layer 520 can be used to add position information for each position of the input text, helping the model understand the semantics and context analysis between the words in the text. The encoding network 530 can have multiple, each of which can be a structure of a transformer stack, used to capture the contextual information in the input text through a self-attention mechanism. The self-attention mechanism allows the model to consider all elements in the sequence when processing each element, thereby better understanding the semantic information in the input text. The linear transformation layer 540 can convert the feature vector output by the encoding network into a probability distribution over the vocabulary through a linear transformation, and convert the vector into a readable text output.

[0104] The target neural network model can be pre-trained, and in an embodiment, as shown in FIG. 5, the target neural network model 500 can be a pre-trained model. Figure 6 As shown, the training process of the target neural network model includes:

[0105] Step 610, obtaining training sample data.

[0106] The training sample data can be applied to the training of the target neural network model, which includes sample text data and a first summary label corresponding to the sample text data.

[0107] The sample text data can include a sample prompt text and a sample to-be-generated text. The sample prompt text is similar to the summary generation prompt text in the foregoing embodiments, and the sample to-be-generated text is similar to the target text in the foregoing embodiments, which will not be described here.

[0108] The first summary label can be the result of the summary generation for the sample text data. In an embodiment, the first summary label can be artificially generated based on the sample text data. That is, the first summary label is obtained by artificially generating a summary for the sample text data.

[0109] In another embodiment, the first summary label can be generated based on an existing large language model. The sample text data is input into the large language model, and based on the guidance of the sample prompt text in the sample text data, the large language model can extract a summary based on the sample to-be-generated text to obtain the first summary label. Generating the first summary label corresponding to the sample text data using the large language model can improve the generation efficiency of the first summary label, and further improve the efficiency of obtaining the training sample data.

[0110] Since the existing large language model has low accuracy in generating summaries. For example, for a script content with complex content, the generated summary may have factual errors or character errors. Therefore, the summary generated by the large language model based on sample text data can be corrected to obtain a more accurate summary generation result, which is used as the first summary label. Data correction can be manually corrected or corrected using an open-source large language model.

[0111] After obtaining the first summary label, the sample text data and the first summary label can be stored correspondingly to obtain training sample data.

[0112] In step 620, the sample text data is input into the preset neural network model to obtain at least one first prediction probability sequence corresponding to at least one first candidate word contained in the first summary label output by the preset neural network model, and a first loss is calculated based on the first summary label and the at least one first prediction probability sequence.

[0113] The preset neural network model can be an open-source neural network model that has been trained with a large number of text samples in general scenarios. Training the preset neural network model into the target neural network model can be a supervised fine-tuning of the preset neural network model based on the summary generation scenario. The target neural network model is obtained by updating and fine-tuning the parameters of the preset neural network model. Therefore, the model architecture of the preset neural network model is the same as that of the target neural network model in the foregoing embodiments, which will not be described again here.

[0114] After obtaining the preset neural network model, the sample text data can be input into the preset neural network model. The preset neural network model can output at least one first prediction probability sequence corresponding to at least one first candidate word contained in the first summary label based on the sample text data. The first prediction probability sequence includes output probabilities of a plurality of reference words in a reference word library.

[0115] The preset neural network model can output a plurality of prediction probability sequences corresponding to a plurality of candidate words. The candidate word can be a word corresponding to each character position in the summary generated by the preset neural network model based on the sample text data. For example, the maximum number of characters in the summary generated by the preset neural network model is 4096, so there are 4096 candidate words, and the preset neural network model can output 4096 prediction probability sequences corresponding to the 4096 candidate words. The prediction probability sequence corresponding to each candidate word can include the output probability of a plurality of reference words in the reference word library at this character position. The reference word can be a word that can appear at each character position. For example, there are 6400 reference words in the reference word library, so the prediction probability sequence corresponding to each candidate word can be a 6400-dimensional vector, which contains the probability of each reference word appearing at this character position. For example Figure 7As shown, in the word position 0001, the probability of the occurrence of the reference word w1 is 0.1, the probability of the occurrence of the reference word w2 is 0.13, the probability of the occurrence of the reference word w3 is 0.21, the probability of the occurrence of the reference word w4 is 0.06, and the probability of the occurrence of the reference word w6400 is 0.17. As can be seen, the preset neural network model can output a two-dimensional array with a row dimension equal to 4096 and a column dimension equal to 6400, each row representing a candidate word, and each column representing a reference word.

[0116] The at least one first candidate word included in the first summary label can be a candidate word corresponding to each word position in the first summary label. For example, if the first summary label contains 15 words, there are 15 first candidate words. The first prediction probability sequence corresponding to the first candidate word can be a prediction probability sequence corresponding to the word position of the first candidate word in the plurality of prediction probability sequences output by the preset neural network model. For example, the plurality of prediction probability sequences shown above, when the first summary label contains 15 words, the first prediction probability sequence can be the prediction probability sequence corresponding to the candidate word 0001-0015. Therefore, the at least one first prediction probability sequence corresponding to the at least one first candidate word can be obtained from the plurality of prediction probability sequences corresponding to the plurality of candidate words output by the preset neural network based on the number of first candidate words in the first summary label. Figure 7

[0117] After obtaining the at least one first prediction probability sequence, the first loss can be calculated based on the first summary label and the at least one first prediction probability sequence. The first loss can measure the difference between the output of the preset neural network model and the first summary label. The smaller the first loss, the higher the accuracy of the summary output by the preset neural network model.

[0118] In an embodiment, the first loss is calculated based on the first summary label and the at least one first prediction probability sequence, comprising:

[0119] For each first candidate word corresponding to the first summary label, a word difference term corresponding to the first candidate word is calculated based on the candidate word label corresponding to the first candidate word in the first summary label and the corresponding first prediction probability sequence;

[0120] The first loss is calculated based on the at least one word difference term corresponding to the at least one first candidate word.

[0121] ​The candidate word label can be a true value corresponding to the first candidate word in the first summary label. For example, the first summary label is "plot content", and then "plot", "content", "emotion", and "internal" are candidate word labels of the first candidate word corresponding to each bit of the first summary label. The word difference item corresponding to the first candidate word can represent the difference between the predicted probability of the preset neural network model for the plurality of reference words corresponding to the first candidate word and the true value corresponding to the first candidate word in the first summary label. The word difference item corresponding to the first candidate word can be obtained by calculating the cross-entropy between the candidate word label corresponding to the first candidate word and the first predicted probability sequence. The calculation formula of the cross-entropy can be represented as formula 1:

[0122]

[0123] In formula 1, l i may represent the word difference item corresponding to the i-th first candidate word in the first summary label; m can represent the number of reference words, and c can represent the serial number of the reference word. y ic may indicate whether the c-th reference word is equal to the candidate word label corresponding to the first candidate word. When the c-th reference word is equal to the candidate word label corresponding to the first candidate word, y ic may be equal to 1; otherwise, when the c-th reference word is not equal to the candidate word label corresponding to the first candidate word, y ic may be equal to 0. p ic may be the predicted probability corresponding to the c-th reference word in the first predicted probability sequence. For example, the candidate word label corresponding to the first candidate word in the first summary label is w2, and the plurality of reference words are w1, w2, w3, and w4 in turn, and the corresponding first predicted probability sequence is [0.1, 0.3, 0.2, 0.4]. Therefore, based on formula 1, the word difference item corresponding to the first candidate word is equal to -(0*log0.1+1*log0.3+0*log0.2+0*log0.4)=0.522.

[0124] After calculating the word difference item corresponding to each first candidate word, the first loss can be calculated based on at least one word difference item corresponding to at least one first candidate word.

[0125] In an implementation, the first loss can be equal to the average value of the at least one word difference item, which can be represented as formula 2:

[0126]

[0127] In formula 2, L1 can represent the first loss, n can represent the number of first candidate words, l iThe word difference item corresponding to the i-th first candidate word can be represented. For example, there are 5 first candidate words, and the corresponding word difference items are 0.522, 0.615, 0.369, 0.537, and 0.414, respectively. The first loss calculated based on formula 2 is equal to (0.522+0.615+0.369+0.537+0.414) / 5=0.4914.

[0128] Measuring the difference between the output of the preset neural network model and the first summary label based on the predicted difference of each first candidate word can ensure that the trained target neural network model can obtain a more accurate generation result for each first candidate word, thereby improving the accuracy of the overall generation of the summary.

[0129] In step 630, a first reference probability sequence corresponding to each first candidate word is obtained, and a second loss is calculated based on the first predicted probability sequence corresponding to each first candidate word and the first reference probability sequence.

[0130] The first reference probability sequence can include reference probabilities of a plurality of reference words corresponding to the first candidate word. The first uniformity of the first probability distribution of the first reference probability sequence is higher than the second uniformity of the second probability distribution of the first predicted probability sequence. The uniformity can be used to indicate the uniformity of the probability distribution of the plurality of reference words. The more similar the probabilities are, the higher the uniformity is. Therefore, the reference probabilities of the plurality of reference words in the first reference probability sequence are relatively similar, and the first uniformity of the first probability distribution is high, which can be used to balance the second probability distribution of the plurality of reference words in the first predicted probability sequence corresponding to the first candidate word, avoiding the problem of overfitting of the model training caused by the predicted probability of the word in the first summary label being too high and the probability of other reference words being too low, and maintaining the diversity of the preset neural network model for generating different summary words for different texts.

[0131] In an embodiment, the first reference probability sequence can be a preset sequence in which the reference probabilities of the plurality of reference words are the same. For example, the reference probability of each reference word in the first reference probability sequence is 1. The first reference probability sequence obtained in this way can ensure that the probability distribution corresponding to each reference word has high uniformity.

[0132] In another embodiment, obtaining the first reference probability sequence corresponding to each first candidate word includes:

[0133] Obtaining a reference neural network model;

[0134] Inputting the sample text data into the reference neural network model to obtain the first reference probability sequence corresponding to each first candidate word.

[0135] In an embodiment, the reference neural network model can be a preset neural network model that is balanced in predicting probability distribution of reference words in other text generation tasks. The preset neural network model can have the same model structure as the preset neural network model. Obtaining the preset neural network model as the reference neural network model can improve the efficiency of obtaining the reference neural network model.

[0136] In another embodiment, the reference neural network model can be a model trained in an iteration round before the current iteration round corresponding to the preset neural network model.

[0137] Training the preset neural network model into the target neural network model needs to go through multiple iterations. With the update iteration of the model parameters, the prediction accuracy of the preset neural network model for multiple reference words improves, the prediction probability for the words in the first summary label increases, and the prediction probability for other reference words decreases. Therefore, with the advancement of the iteration round, the balance of the first prediction probability sequence output by the preset neural network model will gradually decrease. Therefore, the model trained in an iteration round before the current iteration round corresponding to the preset neural network model can be used as the reference neural network model. For example, the model obtained in the previous iteration round corresponding to the preset neural network model currently being trained can be used as the reference neural network model. For example, the current training round is i, and the model used in the i-th training round is the model obtained in the i-1-th iteration, and the model obtained in the i-2-th iteration before the i-1-th iteration can be obtained as the reference neural network model.

[0138] Using the model trained in an iteration round before the current iteration round corresponding to the preset neural network model as the reference neural network model can balance the first probability distribution of the first prediction probability sequence while maintaining the prediction accuracy.

[0139] After obtaining the reference neural network model, the sample text data can be input into the reference neural network model to obtain the first reference probability sequence corresponding to each first candidate word. Obtaining the first reference probability sequence corresponding to each first candidate word based on the reference neural network model can ensure that the first reference probability sequence and the first prediction probability sequence are generated based on the same model structure, which realizes balancing the first reference probability sequence while maintaining the accuracy of the first probability distribution prediction of multiple reference words in the first reference probability sequence.

[0140] After obtaining the first reference probability sequence corresponding to each first candidate word, a second loss can be calculated based on the first predicted probability sequence corresponding to each first candidate word and the first reference probability sequence. The second loss can be used to measure the gap in balance between the first predicted probability sequence and the first reference probability sequence, so as to narrow the first balance of the first probability distribution of the first predicted probability sequence to the second balance of the second probability distribution of the first reference probability sequence.

[0141] In an implementation, the second loss is calculated based on the first predicted probability sequence corresponding to each first candidate word and the first reference probability sequence, including:

[0142] The first difference item corresponding to each first candidate word is calculated based on the first predicted probability sequence corresponding to each first candidate word and the first reference probability sequence.

[0143] The second loss is calculated based on the first difference item.

[0144] The first difference item can indicate the gap between the second probability distribution of the plurality of reference words output by the preset neural network model for each first candidate word and the first probability distribution corresponding to the first reference probability sequence.

[0145] In an implementation, the process of calculating the first difference item corresponding to the first candidate word based on the first predicted probability sequence and the first reference probability sequence can be represented as formula 3:

[0146]

[0147] In formula 3, y w may represent the first summary label; F i may represent the first summary label y w may represent the first difference item corresponding to the i-th first candidate word in the first summary label; x may represent the sample text data input into the preset neural network model; may represent the first predicted probability sequence corresponding to the i-th first candidate word based on the sample text data x, may represent the first reference probability sequence corresponding to the i-th first candidate word. As can be seen, the first difference item can determine the gap between the first probability distribution and the second probability distribution by measuring the amount of additional information generated when the first probability distribution corresponding to the first reference probability sequence approximates the second probability distribution corresponding to the first predicted probability sequence.

[0148] Based on formula 3, the first difference item between the first predicted probability sequence and the first reference probability sequence can be determined by calculating the probability of each reference word in the first predicted probability sequence and the probability gap in the first reference probability sequence, which can be represented as formula 4:

[0149]

[0150] In formula 4, may represent the probability value of the jth reference word in the first prediction probability sequence corresponding to the ith first candidate word. may represent the probability value of the jth reference word in the first reference probability sequence corresponding to the ith first candidate word. By The difference term corresponding to the jth reference word corresponding to the ith first candidate word can be calculated, and the difference terms corresponding to the plurality of reference words are added to obtain the first difference term corresponding to the ith first candidate word.

[0151] For example, the reference words include w1, w2, w3 and w4, the first prediction probability sequence corresponding to the ith first candidate word is [0.1, 0.5, 0.2, 0.2], and the corresponding first reference probability sequence is [0.25, 0.25, 0.25, 0.25]. Based on formula 4, the first difference term corresponding to the first candidate word can be calculated as -(0.1*log(0.1 / 0.25)+0.5*log(0.5 / 0.25)+0.2*log(0.2 / 0.25)+0.2*log(0.2 / 0.25))=0.072.

[0152] After calculating the first difference term corresponding to each first candidate word, the second loss can be calculated based on at least one first difference term corresponding to at least one first candidate word. In an embodiment, the second loss can be obtained by adding at least one first difference term corresponding to at least one first candidate word.

[0153] Calculating the second loss based on the first difference term corresponding to each first candidate word can ensure that the first probability distribution corresponding to each candidate word can achieve a relatively balanced state, improving the comprehensiveness of the prediction balance training of the preset neural network model.

[0154] In another embodiment, the first summary label includes at least two first candidate words. Based on this, before calculating the second loss based on the first difference term, the method further includes:

[0155] determining at least one candidate word sequence based on the at least two first candidate words included in the first summary label;

[0156] calculating a second difference term corresponding to each candidate word sequence based on the first prediction probability sequence and the corresponding first reference probability sequence corresponding to each first candidate word in each candidate word sequence.

[0157] The candidate word sequence can be a sequence of at least two first candidate words adjacent in the first summary label. For example, the first summary label is "plot content", when the candidate word sequence is a sequence of two first candidate words, the candidate word sequence can include "plot", "plot content", and "content".

[0158] In addition to the determination of a single word, the preset neural network model also considers the sequence combination of words or short sentences composed of adjacent words when outputting a sentence. Some sequences have a higher generation probability, while some sequences have a lower generation probability. After learning the first summary label, the preset neural network model can have an overfitting problem that the generation probability of a certain word sequence is high, while the generation probability of other word sequences is low. Therefore, in order to maintain the diversity of the word sequence generated by the preset neural network model, the word sequence generation of the preset neural network model can also be balanced.

[0159] The second difference corresponding to the candidate word sequence can be the balance difference between the prediction probability distribution of the candidate word sequence output by the preset neural network model and the reference probability distribution corresponding to the candidate word sequence. The balance of the reference probability distribution corresponding to the candidate word sequence is higher than the balance of the prediction probability distribution of the candidate word sequence output by the preset neural network model.

[0160] The balance difference between the prediction probability distribution and the reference probability distribution corresponding to the candidate word sequence can be calculated based on the first prediction probability sequence and the first reference probability sequence corresponding to each candidate word in the candidate word sequence.

[0161] In an implementation, the second difference corresponding to each candidate word sequence is calculated based on the first prediction probability sequence and the corresponding first reference probability sequence corresponding to each first candidate word in each candidate word sequence, including:

[0162] The sequence prediction probability distribution is generated based on the first prediction probability sequence corresponding to each first candidate word in each candidate word sequence, and the sequence reference probability distribution is generated based on the first reference probability sequence corresponding to each first candidate word in the candidate word sequence.

[0163] The second difference corresponding to each candidate word sequence is calculated according to the sequence prediction probability distribution and the sequence reference probability distribution corresponding to each candidate word sequence.

[0164] The sequence prediction probability distribution can be a prediction probability distribution corresponding to the candidate word sequence, used to represent the probability distribution of the preset neural network model generating a plurality of reference words for each first candidate word in the candidate word sequence. The sequence prediction probability distribution can be generated based on the first prediction probability sequence corresponding to each first candidate word in the candidate word sequence.

[0165] In an implementation, generating the sequence prediction probability distribution based on the first prediction probability sequence corresponding to each first candidate word in each candidate word sequence comprises:

[0166] concatenating the first prediction probability sequence corresponding to each first candidate word in each candidate word sequence according to the first candidate word arrangement order to obtain a concatenated sequence;

[0167] generating the sequence prediction probability distribution based on the concatenated sequence and the number of first candidate words contained in the candidate word sequence.

[0168] First, the first prediction probability sequence corresponding to each first candidate word in the candidate word sequence can be concatenated according to the first candidate word arrangement order. For example, the candidate word sequence includes two first candidate words, the first prediction probability sequence corresponding to the first first candidate word is [k1, k2, k3, k4], and the first prediction probability sequence corresponding to the second first candidate word is [k5, k6, k7, k8]. The concatenated sequence obtained by concatenating the first prediction probability sequences is [k1, k2, k3, k4, k5, k6, k7, k8].

[0169] After obtaining the concatenated sequence, the sequence prediction probability distribution can be generated based on the concatenated sequence and the number of first candidate words contained in the candidate word sequence. Since the first prediction probability sequence is the probability distribution of the plurality of reference words corresponding to the first candidate word generated by the preset neural network model, the sum of the probability values in the first prediction probability sequence is 1. The sum of the probabilities in the concatenated sequence obtained by concatenating the first prediction probability sequences corresponding to at least two first candidate words is equal to the number of first candidate words. In order to more accurately calculate the probability distribution of the plurality of reference words corresponding to each first candidate word in the candidate word sequence, the probability distribution in the concatenated sequence can be normalized by using the number of first candidate words to obtain a sequence prediction probability distribution with a probability value sum of 1.

[0170] In an implementation, each probability value in the concatenated sequence can be divided by the number of first candidate words, as shown in formula 5:

[0171] S = concat(t1, t2, …, tn) / n (formula 5).

[0172] In formula 5, S represents a sequence prediction probability distribution corresponding to the candidate word sequence; n represents the number of the first candidate words contained in the candidate word sequence; t1, t2, …, tn represent, in turn, a first prediction probability sequence corresponding to each first candidate word in the candidate word sequence; and concat represents an operation of splicing the first prediction probability sequences. For example, the candidate word sequence contains two first candidate words, the first prediction probability sequence of the first first candidate word is [0.1, 0.3, 0.2, 0.4], and the first prediction probability sequence of the second first candidate word is [0.3, 0.1, 0.1, 0.5]. Based on formula 5, the sequence prediction probability distribution corresponding to the candidate word sequence can be obtained as [0.1, 0.3, 0.2, 0.4, 0.3, 0.1, 0.1, 0.5] / 2 = [0.05, 0.15, 0.1, 0.2, 0.15, 0.05, 0.05, 0.25].

[0173] Splicing the first prediction probability sequences corresponding to the at least two first candidate words and generating the sequence prediction probability distribution based on the spliced sequence and the number of the first candidate words can obtain, from the sequence prediction probability distribution, a probability distribution of the multiple reference words corresponding to each first candidate word in the candidate word sequence. Calculating the second difference term corresponding to the entire candidate word sequence with respect to the probability distribution of the multiple reference words corresponding to each first candidate word realizes the balance of the generation probability of the multiple reference word combinations corresponding to the at least two first candidate words, and improves the accuracy of the balanced training of the word sequence of the preset neural network model.

[0174] Similarly, the sequence reference probability distribution can be a reference probability distribution corresponding to the candidate word sequence, and is used to represent the balanced probability distribution of the multiple reference words corresponding to each first candidate word in the candidate word sequence. The sequence reference probability distribution can be generated based on the first reference probability sequence corresponding to each first candidate word in the candidate word sequence, and the generation process can be the same as the process of generating the sequence prediction probability distribution based on the first prediction probability sequence corresponding to each first candidate word in each candidate word sequence in the foregoing embodiments, which will not be described herein again.

[0175] After generating the sequence prediction probability distribution and the sequence reference probability distribution corresponding to each candidate word sequence, the second difference term corresponding to each candidate word sequence can be calculated according to the sequence prediction probability distribution and the sequence reference probability distribution. The calculation manner of calculating the second difference term based on the sequence prediction probability distribution and the sequence reference probability distribution can be the same as the calculation manner of calculating the first difference term corresponding to the first candidate word based on the first prediction probability sequence and the first reference probability sequence of the first candidate word in the foregoing embodiments, and the gap between the sequence prediction probability distribution and the sequence reference probability distribution is determined by measuring the amount of additional information generated when the sequence reference probability distribution approximates the sequence prediction probability distribution.

[0176] The following takes a candidate word sequence composed of two first candidate words as an example to illustrate the calculation process of the second difference item corresponding to the candidate word sequence. The process of calculating the second difference item for the candidate word sequence composed of two first candidate words can be represented as formula 6:

[0177]

[0178] In formula 6, y w may represent the first summary label; F (i,i+1) may represent the first summary label y w may represent the second difference item corresponding to the candidate word sequence composed of the i-th first candidate word and the i+1-th first candidate word; x may represent the sample text data input into the preset neural network model. may represent the first predicted probability sequence corresponding to the i-th first candidate word based on the sample text data x; may represent the first predicted probability sequence corresponding to the i+1-th first candidate word. Therefore, may represent the sequence prediction probability distribution corresponding to the candidate word sequence generated based on the first predicted probability sequence corresponding to the i-th first candidate word and the i+1-th first candidate word. Similarly, may represent the sequence reference probability distribution corresponding to the candidate word sequence generated based on the first reference probability sequence corresponding to the i-th first candidate word and the i+1-th first candidate word.

[0179] The same as the calculation method of the first difference item described in formula 3, the second difference item corresponding to the candidate word sequence composed of two first candidate words is calculated based on formula 6, which will not be described here.

[0180] Based on the first predicted probability sequence corresponding to each first candidate word in the candidate word sequence to generate the sequence prediction probability distribution, and based on the first reference probability sequence corresponding to each first candidate word in the candidate word sequence to generate the sequence reference probability distribution, the balance of the sequence prediction probability distribution and the sequence reference probability distribution corresponding to the candidate word sequence can be determined from the dimension of the first candidate word. According to the calculation of the second difference item based on the sequence prediction probability distribution and the sequence reference probability distribution, the generation probability distribution of the plurality of reference words corresponding to each first candidate word in the candidate word sequence can be ensured to maintain balance, which improves the accuracy of the balance training of the word sequence of the preset neural network model.

[0181] Based on the above implementation, the second loss is calculated based on the first difference item, including: the second loss is calculated based on the first difference item and the second difference item.

[0182] In one implementation, the second loss is calculated based on the first difference item and the second difference item, including:

[0183] The candidate word difference item is obtained by adding the first difference items corresponding to the plurality of first candidate words.

[0184] The candidate word sequence difference item is obtained by adding the second difference items corresponding to the at least one candidate word sequence.

[0185] The second loss is obtained by adding the candidate word difference item and the candidate word sequence difference item.

[0186] When the first difference item corresponding to the first candidate word is calculated according to the above formula 3, the calculation formula of the candidate word difference item can be represented as formula 7:

[0187]

[0188] In formula 7, F seq1 (y w ) can represent the candidate word difference item corresponding to the plurality of first candidate words in the first summary label y w , and n can represent the number of first candidate words in the first summary label. The first difference items corresponding to the n first candidate words are added to obtain the candidate word difference item.

[0189] Taking a candidate word sequence composed of two first candidate words as an example, when the second difference item corresponding to the candidate word sequence is calculated according to the above formula 6, the calculation formula of the candidate word sequence difference item can be represented as formula 8:

[0190]

[0191]

[0192] In formula 8, F seq2 (y w ) can represent the candidate word sequence difference item corresponding to the at least one candidate word sequence in the first summary label y w , and n can represent the number of first candidate words in the first summary label. Each candidate word sequence is composed of two adjacent first candidate words, and n-1 candidate word sequences can be obtained. The second difference items corresponding to the n-1 candidate word sequences are added to obtain the candidate word sequence difference item.

[0193] The second loss is obtained by adding the candidate word difference item and the candidate word sequence difference item. The second loss can be represented as formula 9:

[0194] L2=F seq1 (y w )+F seq2 (y w ) (formula 9).

[0195] The second loss is calculated based on the first difference corresponding to each first candidate word and the second difference corresponding to each candidate word sequence, so that the balance of the preset neural network model can be realized in both the single word dimension and the word sequence dimension, thereby avoiding the output probability of a certain word and a certain word sequence in the first summary label being too high, further maintaining the diversity of the preset neural network model in generating summaries for different texts, and improving the accuracy of the summary generation.

[0196] At step 640, a first target loss is calculated based on the first loss and the second loss, and the parameters of the preset neural network model are updated based on the first target loss, and a target neural network model is determined based on the updated preset neural network model.

[0197] The first loss can be a loss for measuring the accuracy of the output of the preset neural network model, and the second loss can be a loss for measuring the balance of the output of the preset neural network model. Therefore, updating the parameters of the preset neural network model based on the first target loss can realize maintaining the accuracy of the output of the preset neural network model while keeping the diversity of the output of the preset neural network model.

[0198] In an embodiment, the first target loss can be calculated by directly adding the first loss and the second loss, i.e., the first target loss = the first loss + the second loss. For example, the first loss is 1.5 and the second loss is 1, then the first target loss = 1 + 1.5 = 2.5. By directly adding the first loss and the second loss, the influence of the first loss and the second loss on the training of the preset neural network model can be ensured to be the same, thereby improving the fairness of the model accuracy training and the balance training.

[0199] In another embodiment, the first target loss is calculated based on the first loss and the second loss, including:

[0200] The first loss weight corresponding to the first loss is obtained, and the second loss weight corresponding to the second loss is obtained;

[0201] The first loss and the second loss are weighted and calculated based on the first loss weight and the second loss weight, and the first target loss is obtained.

[0202] The first loss weight and the second loss weight can be used to limit the influence of the first loss and the second loss on the training of the preset neural network model. In an embodiment, the first loss weight and the second loss weight can be predetermined values, which can be set according to the degree of accuracy training and balance training of the preset neural network model in actual application. For example, the first loss weight is 0.7 and the second loss weight is 0.3.

[0203] In another implementation, the first loss weight is a predetermined value, and obtaining the second loss weight corresponding to the second loss includes:

[0204] Obtain the weight curve corresponding to the second loss;

[0205] The second loss weight is determined from the weight curve based on the number of iterations in which the parameters of the preset neural network model are updated during the training period.

[0206] The weight curve can be a curve showing the change of the second loss weights as the model is updated and iterated during the training cycle. In one implementation, the weight curve can be a sine function waveform. As the model is updated and iterated, the second loss weights can be updated according to the changes in the weight curve.

[0207] For example Figure 8 As shown, the maximum value of the second loss weight in the weight curve is 1, and the minimum value is 0. During the training period, the model updates 1000 times. In the initial iterations, the second loss weight is 0.5; it then gradually increases to a maximum of 1; subsequently, it gradually decreases, reaching 0 before gradually increasing again; until the 500th iteration, the second loss weight returns to 0.5. From the initial iteration to the 500th iteration, the second loss weight traverses a sine function period (0 degrees - 360 degrees). From the 501st to the 1000th iteration, the second loss weight changes in the same manner.

[0208] Therefore, in each iteration of training, the second loss weight can be determined from the weight curve based on the iteration number. For example, the weight curve is as follows: Figure 8 As shown, when the number of iterations is 500, the weight of the second loss is 0.5.

[0209] The first loss weight can be a predetermined fixed value. The relationship between the first loss weight and the second loss weight can be expressed as follows: Figure 9 .like Figure 9 As shown, the second loss weight changes according to a predetermined weight curve as the number of iterations increases, while the first loss weight uses a fixed value of 0.3 in each iteration.

[0210] The reason for using a fixed value for the first loss weight and updating the second loss weight according to the weight curve is that setting the first loss weight and the second loss weight based on human experience when training the preset neural network model may cause the model accuracy training and the balance training to be unbalanced. When the first loss weight is too large and the second loss weight is too small, the balance training of the model is insufficient, which may cause the model to overfit. When the first loss weight is too small and the second loss weight is too large, the balance of the model output result may be too strong, affecting the accuracy of the model output. The purpose of training the preset neural network model using the first loss and the second loss is to maintain the accuracy of the model output while improving the diversity and balance of the model output, thereby improving the accuracy of the model for generating summaries for different texts. Therefore, when determining the first loss weight and the second loss weight, the first loss weight can be set to a fixed value to maintain the accuracy of the model output, and the second loss weight can be changed based on the weight curve to determine the optimal second loss weight that maintains the balance between the accuracy and balance of the model training.

[0211] Determining the second loss weight based on the weight curve facilitates the evaluation of the influence of different second loss weights on the accuracy training and balance training of the preset neural network model, avoiding problems such as too high balance of the preset neural network model leading to decreased accuracy, or too high accuracy leading to overfitting, thereby improving the balance of the accuracy training and balance training of the preset neural network model.

[0212] After obtaining the first loss weight and the second loss weight, the first loss and the second loss can be weighted and calculated based on the first loss weight and the second loss weight to obtain a first target loss. For example, the first loss weight is 0.7, the second loss weight is 0.3, the first loss is 1.5, and the second loss is 1, then the first target loss is 0.7*1.5+0.3*1=1.35.

[0213] Based on the first loss weight and the second loss weight, the first loss and the second loss are weighted and calculated to obtain the first target loss, which can adjust the degree of accuracy training and balance training of the preset neural network model according to actual training needs, thereby improving the flexibility of the preset neural network model training.

[0214] After calculating the first target loss, the parameters of the preset neural network model can be updated based on the first target loss. When updating the parameters of the preset neural network model, the first target loss can be transmitted back to the preset neural network model, the gradient of each parameter is calculated, and the parameter is updated in the direction of gradient descent. After K rounds of iteration in a training period, a target neural network model is obtained.

[0215] In another implementation, determining the target neural network model based on the updated preset neural network model comprises:

[0216] obtaining a second summary label corresponding to the sample text data;

[0217] inputting the sample text data into the updated preset neural network model to obtain at least one second prediction probability sequence corresponding to at least one first candidate word contained in the first summary label and at least one third prediction probability sequence corresponding to at least one second candidate word contained in the second summary label output by the preset neural network model;

[0218] calculating a third loss based on the first summary label, the second summary label, the second prediction probability sequence, and the third prediction probability sequence;

[0219] obtaining a second reference probability sequence corresponding to each first candidate word, and calculating a fourth loss based on the second prediction probability sequence corresponding to each first candidate word and the second reference probability sequence, and the third prediction probability sequence corresponding to each second candidate word;

[0220] calculating a second target loss according to the third loss and the fourth loss, and updating the parameters of the preset neural network model based on the second target loss to obtain the target neural network model.

[0221] The second summary label can be another summary different from the first summary label obtained based on the sample text data. The second summary label can be obtained in the same way as the first summary label. In one implementation, the second summary label can be artificially generated based on the sample text data. In another implementation, the second summary label can be obtained based on an existing large language model. When the second summary label is generated based on the large language model, the sample prompt text in the sample text data can be different from the sample prompt text when the first summary label is generated, so as to guide the large language model to generate a summary different from the first summary label.

[0222] The first quality score of the first summary label is higher than the second quality score of the second summary label. The quality score can be used to measure whether the corresponding summary is accurate and whether it meets the reading preferences, etc. When determining the first summary label and the second summary label, the two summary labels can be generated artificially or by a large language model, and the quality score of the two summary labels is evaluated, the summary label with a higher quality score is determined as the first summary label, and the summary label with a lower quality score is determined as the second summary label. The quality score of the summary label can be evaluated by comparing the two summary labels artificially to determine which summary label has a higher quality; or the quality score of the summary label can be evaluated by a large language model.

[0223] The sample text data, the first summary label and the second summary label jointly constitute training sample data, so as to further train the preset neural network model.

[0224] The sample text data is input into the preset neural network model trained by the foregoing steps 610-640 again, to obtain at least one second prediction probability sequence corresponding to at least one first candidate word contained in the first summary label and at least one third prediction probability sequence corresponding to at least one second candidate word contained in the second summary label output by the preset neural network model. The second candidate word indicates a candidate word corresponding to each bit of the second summary label. The second prediction probability sequence and the third prediction probability sequence are similar to the first prediction probability sequence in the foregoing embodiment, the second prediction probability sequence contains prediction probabilities of the plurality of reference words corresponding to the first candidate word output by the updated preset neural network model, and the third prediction probability sequence contains prediction probabilities of the plurality of reference words corresponding to the second candidate word.

[0225] After obtaining the second prediction probability sequence and the third prediction probability sequence, a third loss can be calculated based on the first summary label, the second summary label, the second prediction probability sequence and the third prediction probability sequence.

[0226] In an embodiment, the third loss is calculated based on the first summary label, the second summary label, the second prediction probability sequence and the third prediction probability sequence, including:

[0227] The first generation probability of the first summary label generated by the preset neural network model is determined based on the at least one second prediction probability sequence;

[0228] The second generation probability of the second summary label generated by the preset neural network model is determined based on the at least one third prediction probability sequence;

[0229] The third loss is calculated based on the first generation probability and the second generation probability.

[0230] The second prediction probability sequence indicates a generation probability distribution of the plurality of reference words corresponding to the first candidate word in the first summary label. Therefore, in an embodiment, the first generation probability of the first summary label generated by the preset neural network model is determined based on the at least one second prediction probability sequence, including:

[0231] The candidate word label generation probability is determined based on the second prediction probability sequence corresponding to each first candidate word and the corresponding candidate word label in the first summary label;

[0232] The first generation probability is calculated based on at least one candidate word label generation probability corresponding to at least one first candidate word.

[0233] The candidate word label can be the true value corresponding to the first candidate word in the first summary label, as described in detail in the preceding embodiments, and will not be repeated here. The second prediction probability sequence contains the probability corresponding to the candidate word label. Therefore, the probability corresponding to the candidate word label can be directly obtained from the second prediction probability sequence as the candidate word label generation probability. For example, if the candidate word label corresponding to a first candidate word in the first summary label is w1, and the multiple reference words are w1, w2, w3, and w4 respectively, the second prediction probability sequence corresponding to the first candidate word is [0.3, 0.2, 0.1, 0.4]. Therefore, the candidate word label generation probability corresponding to candidate word label w1 is 0.3.

[0234] The first generation probability is calculated based on the generation probability of at least one candidate word tag corresponding to at least one first candidate word. This can be achieved by multiplying the generation probabilities of at least one candidate word tag to obtain the first generation probability corresponding to the first summary tag. For example, if the first summary tag is "plot content", based on the second prediction probability sequence corresponding to the four first candidate words, the generation probabilities of the preset neural network model generating candidate word tags for "drama", "plot", "content", and "containment" in these four positions are 0.3, 0.5, 0.2, and 0.3, respectively. Therefore, the first generation probability of the first summary tag is 0.3 * 0.5 * 0.2 * 0.3 = 0.009.

[0235] The process of determining the second generation probability of the second summary tag based on at least one third prediction probability sequence is the same as the process of determining the first generation probability, and will not be repeated here.

[0236] After determining the first generation probability and the second generation probability, a third loss can be calculated based on the first generation probability and the second generation probability. In one implementation, the process of calculating the third loss based on the first generation probability and the second generation probability can be expressed as Equation 10:

[0237]

[0238] In Formula 10, Let x, y1, and y2 represent the training sample data. x can represent sample text data, y1 can represent the first summary label, and y2 can represent the second summary label. P(y1|x) can represent the first generation probability of generating the first summary label based on the sample text data; P(y2|x) can represent the second generation probability of generating the second summary label based on the sample text data. σ can represent a logistic function used to normalize the difference between the first and second generation probabilities. Then, based on the log function, the result of the normalization is mapped to a smaller range to obtain the third loss L3.

[0239] In the training of the preset neural network model based on the third loss obtained from formula 10, in order to reduce the third loss, the training target can be to increase the distance between the first generation probability and the second generation probability, that is, to increase the first generation probability of the preset neural network model after parameter update to generate the first summary label, and to reduce the second generation probability of the preset neural network model after parameter update to generate the second summary label, so that the target neural network model can generate a text summary with a higher quality score.

[0240] In another embodiment, before calculating the third loss based on the first generation probability and the second generation probability, the method further comprises: obtaining a first generation reference probability corresponding to the first summary label and a second generation reference probability corresponding to the second summary label. Therefore, calculating the third loss based on the first generation probability and the second generation probability comprises: calculating the third loss based on the first generation probability, the second generation probability, the first generation reference probability, and the second generation reference probability.

[0241] The first generation reference probability can be the probability of a model generating the first summary label with relatively high accuracy, and the second generation reference probability can be the probability of the model generating the second summary label. The model with relatively high accuracy in generating summaries can be the preset neural network model obtained by updating the parameters based on the first target loss in step 640. Therefore, the sample text data can be input into the preset neural network model obtained in step 640 to obtain the first generation reference probability and the second generation reference probability.

[0242] For a model with relatively high accuracy in generating summaries, the first generation reference probability of generating the first summary label with a relatively high quality score will be greater than the second generation reference probability of generating the second summary label. In order to enable the model to further improve the probability of generating the first summary label and reduce the probability of generating the second summary label on this basis, the parameters of the preset neural network model can be updated to further improve the first generation probability based on the first generation reference probability.

[0243] In one embodiment, the process of calculating the third loss based on the first generation probability, the second generation probability, the first generation reference probability, and the second generation reference probability can be represented as formula 11:

[0244]

[0245] In formula 11, Q(y1|x) can represent a first generation reference probability of generating the first summary label y1 based on the sample text data x; Q(y2|x) can represent a second generation reference probability of generating the second summary label y2 based on the sample text data x. β can be a parameter used to control the degree of deviation between the first generation probability and the first generation reference probability, and between the second generation probability and the second generation reference probability. By setting β, the influence of the first generation reference probability and the second generation reference probability on the first generation probability and the second generation probability can be controlled. The meanings represented by other parameters are the same as those in formula 10, which will not be repeated here.

[0246] When training the preset neural network model based on the third loss obtained from formula 11, the training target can be to increase the difference between the first generation probability and the first generation reference probability, and to decrease the difference between the second generation probability and the second generation reference probability, so as to make the value of increase, and the value of decrease, thereby increasing the distance between the two values, and further reducing the third loss. In this way, the probability of the preset neural network model tending to generate the first summary label is increased, and the summary generation accuracy of the preset neural network model is improved.

[0247] Calculating the third loss based on the first generation probability, the second generation probability, the first generation reference probability and the second generation reference probability can further improve the accuracy of summary generation on the basis of a summary generation model that is relatively accurate.

[0248] After calculating the third loss, the second reference probability sequence corresponding to each first candidate word can be obtained, and a fourth loss can be calculated based on the second prediction probability sequence corresponding to each first candidate word and the second reference probability sequence, and the third prediction probability sequence corresponding to each second candidate word.

[0249] The second reference probability sequence corresponding to the first candidate word is similar to the first reference probability sequence in the foregoing step, and only the output probability corresponding to each reference word in the second reference probability sequence is different in different training scenarios. Therefore, the uniformity of the second reference probability sequence is higher than that of the second prediction probability sequence. The acquisition method of the second reference probability sequence is the same as that of the first reference probability sequence, which will not be repeated here.

[0250] The fourth loss can be used to measure the difference between the second predicted probability sequence and the second reference probability sequence when the first summary label is output, and the predicted probability distribution corresponding to each second candidate word when the second summary label is output. The third loss can increase the probability of the preset neural network model generating a first summary label with a high quality score. Therefore, balanced training of the preset neural network model based on the fourth loss can avoid overfitting problems of the target neural network model, such as too high probability of outputting a word in the first summary label and too low probability of outputting a word in the second summary label, thereby maintaining the balance of the probability distribution of the preset neural network generating multiple reference words.

[0251] In an embodiment, the fourth loss is calculated based on the second predicted probability sequence corresponding to each first candidate word and the second reference probability sequence, and the third predicted probability sequence corresponding to each second candidate word, comprising:

[0252] calculating a first candidate word difference item corresponding to the first summary label based on the second predicted probability sequence corresponding to each first candidate word and the second reference probability sequence;

[0253] calculating a second candidate word difference item corresponding to the second summary label based on the third predicted probability sequence corresponding to each second candidate word;

[0254] calculating the fourth loss according to the first candidate word difference item and the second candidate word difference item.

[0255] The first candidate word difference item corresponding to the first summary label can be calculated based on the second predicted probability sequence corresponding to each first candidate word and the second reference probability sequence. The specific calculation process can be the same as the process of calculating the second loss based on the first predicted probability sequence corresponding to each first candidate word and the first reference probability sequence in the aforementioned step 630. In an embodiment, the first difference item corresponding to each first candidate word can be calculated by calculating the second predicted probability sequence corresponding to each first candidate word and the second reference probability sequence, and the first candidate word difference item corresponding to the first summary label can be calculated based on the first difference item. In another embodiment, the first summary label contains at least two first candidate words, and at least one candidate word sequence can be determined based on the at least two first candidate words. The second difference item corresponding to each candidate word sequence can be calculated based on the second predicted probability sequence corresponding to each first candidate word in each candidate word sequence and the second reference probability sequence, and the first candidate word difference item corresponding to the first summary label can be calculated based on the first difference item and the second difference item. The detailed process has been described in the aforementioned embodiment, which will not be described here.

[0256] The second candidate word difference item corresponding to the second summary label can be calculated based on the third predicted probability sequence corresponding to each second candidate word. The second candidate word difference item can be calculated based on the difference item corresponding to each second candidate word.

[0257] In an implementation, the calculation process of the difference item corresponding to each second candidate word can be represented as formula 12:

[0258]

[0259] In formula 12, G i may represent the difference item corresponding to the i-th second candidate word in the second summary label; may represent the third prediction probability sequence corresponding to the i-th second candidate word in the second summary label y2 generated based on the preset neural network model based on the sample text data x. Therefore, the difference item corresponding to the second candidate word is equal to the sum of the information entropy corresponding to each reference word in the third prediction probability sequence. When the output probabilities corresponding to all reference words are equal, the probability distribution of the third prediction probability sequence is uniform, and at this time, the difference item corresponding to the second candidate word reaches the maximum value.

[0260] The process of calculating the second candidate word difference item based on the difference item corresponding to each second candidate word can be represented as formula 13:

[0261]

[0262] In formula 13, F en (y2) can represent the second candidate word difference item corresponding to the second summary label y2; n can represent the number of second candidate words. By adding the difference item corresponding to each second candidate word, the second candidate word difference item corresponding to the second summary label can be obtained.

[0263] After obtaining the first candidate word difference item and the second candidate word difference item, the fourth loss can be calculated based on the first candidate word difference item and the second candidate word difference item.

[0264] In an implementation, the fourth loss is calculated according to the first candidate word difference item and the second candidate word difference item, including:

[0265] determining a total difference item based on the first candidate word difference item and the second candidate word difference item;

[0266] normalizing the total difference item to obtain the fourth loss.

[0267] In an implementation, the total difference item can be obtained by summing the first candidate word difference item and the second candidate word difference item.

[0268] In another implementation, the total difference item is determined based on the first candidate word difference item and the second candidate word difference item, including:

[0269] obtaining a first adjustment parameter corresponding to the first candidate word difference item and a second adjustment parameter corresponding to the second candidate word difference item;

[0270] calculating a total difference item based on the first adjustment parameter, the second adjustment parameter, the first candidate word difference item and the second candidate word difference item.

[0271] The first adjustment parameter and the second adjustment parameter can set the influence of the first summary label and the second summary label on the preset neural network model training based on the first adjustment parameter and the second adjustment parameter. When the influence of the first summary label and the second summary label on the preset neural network model training is required to be the same, the first adjustment parameter and the second adjustment parameter can be set to the same value, such as 0.5. The first adjustment parameter and the second adjustment parameter can be pre-set based on the requirements in actual training or training experience.

[0272] After determining the first adjustment parameter and the second adjustment parameter, the total difference item can be calculated based on the first adjustment parameter, the second adjustment parameter, the first candidate word difference item and the second candidate word difference item, which can be represented as formula 14:

[0273] T = β1*F seq (y1) + β2*F en (y2) (formula 14).

[0274] In formula 14, T can represent the total difference item; β1 can represent the first adjustment parameter; β2 can represent the second adjustment parameter; F seq (y1) can represent the first candidate word difference item based on the first summary label y1; F en (y2) can represent the second candidate word difference item based on the second summary label y2.

[0275] Based on the first adjustment parameter and the second adjustment parameter, the first candidate word difference item and the second candidate word difference item are weighted and calculated to obtain the total difference item. The influence of the first summary label and the second summary label on the preset neural network model training can be adjusted according to the requirements of the training in actual application, which improves the flexibility of the preset neural network model training.

[0276] After calculating the total difference item, the total difference item can be normalized to obtain a fourth loss. Specifically, the calculation process of the fourth loss can be represented as formula 15:

[0277]

[0278] In formula 15, The training sample data composed of x, y1 and y2 can be represented, x can represent sample text data, y1 can represent a first summary label, and y2 can represent a second summary label. T can represent a total difference term, and sigma can represent a logistic function for normalizing the total difference term, and mapping the normalized result to a smaller range based on the log function to obtain a fourth loss L4.

[0279] In the foregoing embodiment, the first difference term corresponding to each first candidate word can be calculated based on the second prediction probability sequence and the second reference probability sequence corresponding to each first candidate word. The second difference term corresponding to each candidate word sequence can be calculated based on the second prediction probability sequence and the second reference probability sequence corresponding to each first candidate word in each candidate word sequence, and the first candidate word difference term corresponding to the first summary label can be calculated based on the first difference term and the second difference term. In another embodiment, the total difference term is calculated based on the first adjustment parameter and the second adjustment parameter. Therefore, in an embodiment, the calculation process of the fourth loss can be represented as formula 16:

[0280]

[0281] In formula 16, F seq1 (y1) can represent the sum of the plurality of first difference terms corresponding to the plurality of first candidate words in the first summary label, and F seq2 (y1) can represent the sum of the at least one second difference term corresponding to the at least one candidate word sequence in the first summary label, and the sum of the plurality of first difference terms and the sum of the at least one second difference term constitute the first candidate word difference term; based on the first adjustment parameter β1, the second adjustment parameter β2, the first candidate word difference term and the second candidate word difference term F en (y2), the total difference term can be obtained.

[0282] The calculation of the fourth loss based on the first candidate word difference term corresponding to each first candidate word and the second candidate word difference term corresponding to each second candidate word realizes the balanced training for the first summary label and the second summary label respectively, and improves the accuracy of balanced training of the preset neural network model.

[0283] After the fourth loss is calculated, the second target loss can be calculated according to the third loss and the fourth loss. The calculation method of the second target loss can be the same as the process of calculating the first target loss according to the first loss and the second loss in the foregoing step 640, which will not be described here.

[0284] After the first loss is calculated, the parameters of the preset neural network model can be updated based on the first target loss. In a training period, based on the parameter update of the predetermined number of iterations, the target neural network model can be obtained.

[0285] Based on the further training of the preset neural network model based on the first summary label and the second summary label, the reinforcement training of the preset neural network model can be realized. By comparing the quality of the first summary label and the second summary label, the accuracy of the preset neural network model in generating a summary is further improved. At the same time, in the process of reinforcement training, the second reference probability sequence with higher balance is used to maintain the diversity of the summary generation of the preset neural network model, avoiding the problem of overfitting of the preset neural network model to the first summary label after reinforcement training. Therefore, through the present embodiment, the target neural network model summary obtained by training can output a summary with higher accuracy for different texts.

[0286] In summary, when training the target neural network model for generating a text summary, the text summary generation method of the present embodiment can obtain a more accurate result when the target neural network model generates a text summary by using the first loss calculated based on the at least one first predicted probability sequence corresponding to the first label and the at least one first candidate word. In addition, the second loss is calculated based on the first predicted probability sequence corresponding to each first candidate word and the reference probability sequence. Since the first balance of the first probability distribution of the reference probability sequence is higher than the second balance of the second probability distribution of the first predicted probability sequence, the target neural network model can also have higher balance when generating the first predicted probability sequence corresponding to each first candidate word based on the second loss. In this way, the output probability of some reference words will not be too high and the output probability of other reference words will not be too low due to the guidance of the first label on the generation result. Therefore, the target neural network model after training can ensure the accuracy of text summary generation while avoiding overfitting to the first training sample data, maintaining the diversity of text summary generation, achieving precise word selection for the text summary corresponding to the target text, and improving the accuracy of text summary generation.

[0287] The present embodiment is combined with the detailed description of a specific application scenario

[0288] As Figure 10 shown, the text summary generation method provided by the present disclosure is applied to a specific process diagram of a script summary generation scene, and the text summary generation method comprises:

[0289] Step 1001, obtaining sample plot data and a plurality of sample prompt texts, and inputting each sample prompt text and the sample plot data into a large language model to obtain a candidate summary label.

[0290] The sample prompt text can be a text guiding the script summary generation task of the model. The summary generation prompt text can include a requirement for plot summary generation, so that the model generates a plot summary according to the summary generation requirement. For example, the sample prompt text is: "Based on the given script text, give a plot summary. Requirements: describe the events, time, place, cause, process, and result in the script, and the length is within 80 words."

[0291] The sample plot data is input into the large language model. Based on the guidance of the sample prompt text, the large language model can extract the summary based on the sample plot data to obtain a candidate summary label.

[0292] Since the existing large language model has low accuracy in generating summaries, the summary generated by the large language model can be corrected to obtain a more accurate candidate summary label. Data correction can be manually corrected or corrected using an open-source large language model.

[0293] Different sample prompt texts can generate different plot summaries for the same sample plot data. For example, another sample prompt text is: "Please give a plot summary of the following script text. Requirements: describe the events in the script, and the length is within 80 words." Therefore, based on multiple sample prompt texts, multiple candidate summary labels corresponding to the sample plot data can be obtained.

[0294] Step 1002, determining a first summary label and a second summary label from the candidate summary labels.

[0295] The first quality score of the first summary label is greater than the second quality score of the second summary label. The quality score can be used to measure whether the corresponding summary is accurate and whether it meets the reading preferences, etc. In determining the first summary label and the second summary label, the quality scores of the multiple candidate summary labels can be determined by human comparison or a large language model, and the first summary label and the second summary label can be determined based on the quality scores.

[0296] After determining the first summary label and the second summary label, the sample plot data and the first summary label can be stored as first training data, and the sample plot data, the first summary label, and the second summary label can be stored as second training data.

[0297] Step 1003, inputting the sample plot data in the first training data into a preset neural network model to obtain multiple first prediction probability sequences corresponding to multiple first candidate words included in the first summary label.

[0298] The preset neural network model can be an open source neural network model that has been trained using a large number of text samples in general scenarios. Training the preset neural network model into the target neural network model can be based on the script summary generation scenario to supervise the fine-tuning of the preset neural network model. The preset neural network model can include a text encoding layer, a position encoding layer, an encoding network, and a linear transformation layer. The text encoding layer can be used to encode the input text into a fixed-length context vector that captures the context semantic information in the input text. The position encoding layer can be used to add position information to each position of the input text to help the model understand the semantics and context analysis between the words in the text. The position encoding layer can input a 6400-dimensional vector and output a 4096-dimensional vector. The encoding network can be a transformer stack structure used to capture the context information in the input text through a self-attention mechanism. The encoding network receives a 4096-dimensional vector and then outputs a 6400-dimensional vector. The linear transformation layer can convert the feature vector output by the encoding network into a probability distribution on the vocabulary through a linear transformation, and convert the vector into readable text output.

[0299] The sample text data is input into the preset neural network model. The preset neural network model can output, based on the sample script data, a plurality of first predicted probability sequences corresponding to a plurality of first candidate words contained in the first summary label. The first predicted probability sequence includes an output probability corresponding to a plurality of reference words in a reference word library. The first candidate word can indicate a word corresponding to each position in the first summary label. The reference word can indicate a word that can be generated for each position. Therefore, the first predicted probability sequence indicates the probability distribution of the word that can be generated for each position in the first summary label.

[0300] Step 1004, calculating a first loss based on the first summary label and the plurality of first predicted probability sequences.

[0301] In calculating the first loss based on the first summary label and the plurality of first predicted probability sequences, first, for each first candidate word in the first summary label, a word difference term corresponding to the first candidate word can be calculated based on the candidate word label corresponding to the first candidate word in the first summary label and the corresponding first predicted probability sequence.

[0302] The candidate word label can be the true value of the first candidate word in the first summary label. Based on the cross entropy between the probability distribution in the first predicted probability sequence and the candidate word, the word difference term corresponding to the first candidate word can be calculated, as shown in formula 1 above.

[0303] After calculating the word difference term corresponding to each first candidate word, the average of the plurality of word difference terms corresponding to the plurality of first candidate words can be calculated to determine the first loss.

[0304] In step 1005, a first reference probability sequence corresponding to each first candidate word is obtained, and a second loss is calculated based on the first prediction probability sequence corresponding to each first candidate word and the first reference probability sequence.

[0305] The first reference probability sequence can include reference probabilities of a plurality of reference words corresponding to the first candidate word. A first uniformity of a first probability distribution of the first reference probability sequence is higher than a second uniformity of a second probability distribution of the first prediction probability sequence. The reference probabilities of the plurality of reference words in the first reference probability sequence are relatively close, and the first uniformity of the first probability distribution is relatively high, which can be used to balance the second probability distribution of the plurality of reference words in the first prediction probability sequence corresponding to the first candidate word.

[0306] The first reference probability sequence can be generated by a reference neural network model based on sample plot data. The reference neural network model can be a preset neural network model that is relatively balanced in predicting probability distribution of reference words in other text generation tasks, or a model trained in previous iteration rounds before the current iteration round corresponding to the preset neural network model. The first reference probability sequence can also be a preset probability sequence, for example, the reference probability of each reference word in the first reference probability sequence is 1.

[0307] In calculating the second loss based on the first prediction probability sequence corresponding to each first candidate word and the first reference probability sequence, a first difference term corresponding to each first candidate word can be calculated based on the first prediction probability sequence corresponding to each first candidate word and the first reference probability sequence. The first difference term can indicate the gap between the second probability distribution of the plurality of reference words output by the preset neural network model for each first candidate word and the first probability distribution corresponding to the first reference probability sequence. The process of calculating the first difference term can be represented as formula 3. Adding the plurality of first difference terms corresponding to the plurality of first candidate words can obtain a candidate word difference term, as shown in formula 7.

[0308] Secondly, at least one candidate word sequence can be determined based on the plurality of first candidate words contained in the first summary label. The candidate word sequence can be a sequence formed by two adjacent first candidate words in the first summary label.

[0309] A second difference term corresponding to each first candidate word sequence can be calculated based on the first prediction probability sequence corresponding to each first candidate word in each candidate word sequence and the corresponding first reference probability sequence. The calculation process of the second difference term can be represented as formula 6. Adding the plurality of second difference terms corresponding to the plurality of candidate word sequences can obtain a candidate word sequence difference term, as shown in formula 8.

[0310] After obtaining the candidate word difference term and the candidate word sequence difference term, the candidate word difference term and the candidate word sequence difference term can be added to obtain the second loss, as shown in formula 9.

[0311] Step 1006, a first target loss is calculated according to the first loss and the second loss, and parameters of the preset neural network model are updated based on the first target loss.

[0312] A first loss weight corresponding to the first loss and a second loss weight corresponding to the second loss are obtained. The first loss weight can be a predetermined value, and the second loss weight can be determined based on a weight curve and a model iteration round. The weight curve can be a curve of change of the second loss weight with the advancement of model update iterations in a training period, and can be a sin function waveform curve. The second loss weight is determined in the weight curve based on a current iteration round of the preset neural network model in the training period.

[0313] The first target loss can be obtained by weighting and calculating the first loss and the second loss using the first loss weight and the second loss weight. After the first target loss is calculated, the parameters of the preset neural network model can be updated based on the first target loss. When updating the parameters of the preset neural network model, the first target loss can be fed back to the preset neural network model, the gradient of each parameter can be calculated, and the parameter can be updated in the direction of gradient descent.

[0314] Step 1007, after the parameters of the preset neural network model are updated for a first predetermined iteration round, a first target neural network model is obtained, and a first generated reference probability and a second generated reference probability are obtained based on the first target neural network model.

[0315] After the first predetermined iteration round of update in a training period, the first target neural network model is obtained.

[0316] The first generated reference probability can be a probability of the first target neural network model outputting the first summary label; and the second generated reference probability can be a probability of the first target neural network model outputting the second summary label. The sample plot data is input into the first target neural network model again to obtain the first generated reference probability and the second generated reference probability. The first generated reference probability and the second generated reference probability can be used in subsequent training processes.

[0317] Step 1008, sample plot data in the second training data is input into the first target neural network model to obtain a plurality of second prediction probability sequences corresponding to a plurality of first candidate words contained in the first summary label, and a plurality of third prediction probability sequences corresponding to a plurality of second candidate words contained in the second summary label.

[0318] The first candidate word can indicate a word corresponding to each bit in the first summary label. The second candidate word can indicate a word corresponding to each bit in the second summary label. The first candidate word corresponds to a plurality of reference words, and the second candidate word corresponds to a plurality of reference words. The plurality of reference words are output by the preset neural network model.

[0319] In obtaining the second prediction probability sequence, the prediction probability sequence of each bit corresponding to the first summary label can be obtained from the prediction probability sequence corresponding to the plurality of bits output by the first target neural network model. The third prediction probability sequence can be obtained in the same way.

[0320] Step 1009, calculate a first generation probability based on the first summary label and the second prediction probability sequence, and calculate a second generation probability based on the second summary label and the third prediction probability sequence.

[0321] The first generation probability can be the probability of the first target neural network model outputting the first summary label in the current iteration round; the second generation probability can be the probability of the first target neural network model outputting the second summary label in the current iteration round.

[0322] In calculating the first generation probability, the candidate word label generation probability can be determined based on the second prediction probability sequence corresponding to each first candidate word and the corresponding candidate word label in the first summary label; and the candidate word label generation probabilities corresponding to the plurality of first candidate words are multiplied to obtain the first generation probability corresponding to the first summary label. The second generation probability corresponding to the second summary label can be calculated in the same way.

[0323] Step 1010, calculate a third loss based on the first generation probability, the second generation probability, the first generation reference probability, and the second generation reference probability.

[0324] After obtaining the first generation probability and the second generation probability, the third loss can be calculated based on the first generation probability, the second generation probability, the first generation reference probability, and the second generation reference probability. The first generation reference probability and the second generation reference probability are obtained by the first target neural network model based on the sample plot data in step 1007. The calculation process of the third loss can be represented as formula 11.

[0325] Step 1011, obtain a second reference probability sequence corresponding to each first candidate word, and calculate a fourth loss based on the second prediction probability sequence corresponding to each first candidate word and the second reference probability sequence, and the third prediction probability sequence corresponding to each second candidate word.

[0326] The second reference probability sequence can be the same as or different from the first reference probability sequence. The balance of the second reference probability sequence is higher than that of the second predicted probability sequence. The second reference probability sequence can be pre-set or generated by a reference neural network model based on sample plot data. The reference neural network model can be a pre-set neural network model that has a relatively balanced distribution of predicted probability of reference words in other text generation tasks; it can be a model trained in previous iterations; or it can be the first target neural network model in step 1007.

[0327] When the reference neural network model is the first target neural network model trained in step 1007, both the third and fourth losses depend on the output of this model. For example... Figure 11 As shown, the first target neural network model 1110 in the current iteration has the same structure as the reference neural network model 1120. The first target neural network model 1110 in the current iteration includes a text encoding layer 1111, an embedding layer 1112, multiple encoding networks 1113, and a linear transformation layer 1114. The reference neural network model 1120 includes a text encoding layer 1121, an embedding layer 1122, multiple encoding networks 1123, and a linear transformation layer 1124. A third loss is obtained based on the first generation probability of the first summary label and the second generation probability of the second summary label output by the first target neural network model 1110 corresponding to the current iteration, as well as the first generation reference probability of the first summary label and the second generation reference probability of the second summary label output by the reference neural network model 1120. A fourth loss is calculated based on the second predicted probability sequence and the third predicted probability sequence of the first summary label obtained by the encoding network 1113 of the first target neural network model 1110 corresponding to the current iteration, as well as the second reference probability sequence of the first summary label obtained by the encoding network 1123 of the reference neural network model 1120.

[0328] When calculating the fourth loss, the difference term of the first candidate word corresponding to the first summary label can be calculated firstly based on the second predicted probability sequence and the second reference probability sequence corresponding to each first candidate word. The calculation process of the first candidate word difference term is the same as the calculation process of the second loss in step 1005.

[0329] Then, based on the third predicted probability sequence corresponding to each second candidate word, the difference term of the second candidate word corresponding to the second summary tag is calculated. The calculation process of the difference term of the second candidate word can be expressed as Formula 13 above.

[0330] After obtaining the first candidate word difference item and the second candidate word difference item, the first adjustment parameter corresponding to the first candidate word difference item and the second adjustment parameter corresponding to the second candidate word difference item can be obtained. As shown in the above formula 14, the total difference item is calculated based on the first adjustment parameter, the second adjustment parameter, the first candidate word difference item and the second candidate word difference item. Finally, the fourth loss is obtained by normalizing the total difference item. Therefore, the process of calculating the fourth loss based on the first candidate word difference item and the second candidate word difference item can be represented as the above formula 16.

[0331] Step 1012, calculate the second target loss according to the third loss and the fourth loss, and update the parameters of the first target neural network model based on the second target loss. After the update of the second predetermined iteration round, the second target neural network model is obtained.

[0332] The calculation method of the second target loss is the same as the calculation process of the first target loss in step 1006. The third loss weight corresponding to the third loss and the fourth loss weight corresponding to the fourth loss can be obtained. The third loss weight can be a predetermined value, and the fourth loss weight can be determined based on the weight curve and the model iteration round. The weight curve can also be a sin function waveform curve. The fourth loss weight is determined in the weight curve based on the current iteration round of the first target neural network model in the training period.

[0333] The third loss and the fourth loss are weighted and calculated by using the third loss weight and the fourth loss weight, and the second target loss can be obtained. After the second target loss is calculated, the parameters of the first target neural network model can be updated based on the second target loss. When updating the parameters of the first target neural network model, the second target loss can be fed back to the first target neural network model, the gradient of each parameter is calculated, and the parameter is updated in the direction of gradient descent. After the update of the second predetermined iteration round in a training period, the second target neural network model is obtained.

[0334] Step 1013, obtain the target script plot and the summary generation prompt text; input the target script plot and the summary generation prompt text into the second target neural network model to obtain the target plot summary.

[0335] The target script plot can be a plot text that needs to be summarized. The summary generation prompt text can be a text that provides guidance for the summary generation task of the model. The target script plot is input into the second target neural network model, and the target plot summary corresponding to the target script plot can be generated.

[0336] Apparatus and device description of embodiments of the present disclosure

[0337] It can be understood that, although each step in each of the above flowcharts is shown in sequence according to the representation of the arrow, these steps are not necessarily executed in the order represented by the arrow. Unless otherwise specified in the embodiments, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least part of the steps in the above flowcharts can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or steps or stages in other steps.

[0338] It should be noted that, in each specific embodiment of the present application, when it is necessary to perform relevant processing according to data related to the characteristics of the target content, such as target content attribute information or attribute information set, the permission or consent of the target content is obtained first, and the collection, use and processing of these data comply with relevant laws, regulations and standards. In addition, when the embodiments of the present application need to obtain target content attribute information, a separate permission or separate consent of the target content is obtained through a pop-up window or a jump to a confirmation page, and after obtaining the separate permission or separate consent of the target content, the necessary target content related data for enabling the embodiments of the present application to normally run is obtained.

[0339] Figure 12 A structure diagram of a text summary generation apparatus 1200 provided by the embodiments of the present disclosure is provided. The text summary generation apparatus 1200 comprises:

[0340] An acquisition unit 1210 is configured to acquire a target text and a summary generation prompt text;

[0341] A generation unit 1220 is configured to input the target text and the summary generation prompt text into a pre-trained target neural network model to obtain a text summary corresponding to the target text;

[0342] The training process of the target neural network model comprises:

[0343] Acquiring training sample data, the training sample data comprising sample text data and a first summary label corresponding to the sample text data;

[0344] Inputting the sample text data into a preset neural network model to obtain at least one first prediction probability sequence corresponding to at least one first candidate word contained in the first summary label output by the preset neural network model, the first prediction probability sequence comprising output probabilities corresponding to a plurality of reference words in a reference word library, and calculating a first loss based on the first summary label and the at least one first prediction probability sequence;

[0345] obtain a first reference probability sequence corresponding to each first candidate word, and calculate a second loss based on the first prediction probability sequence corresponding to each first candidate word and the first reference probability sequence, a first probability distribution corresponding to the first reference probability sequence having a higher first balance than a second probability distribution corresponding to the first prediction probability sequence;

[0346] calculate a first target loss according to the first loss and the second loss, update parameters of the preset neural network model based on the first target loss, and determine a target neural network model based on the updated preset neural network model.

[0347] Optionally, in an embodiment, the second loss is calculated based on the first prediction probability sequence corresponding to each first candidate word and the first reference probability sequence, including:

[0348] calculate a first difference term corresponding to each first candidate word based on the first prediction probability sequence corresponding to each first candidate word and the first reference probability sequence;

[0349] calculate the second loss based on the first difference term.

[0350] Optionally, in an embodiment, the first summary label includes at least two first candidate words, and before the second loss is calculated based on the first difference term, further including:

[0351] determine at least one candidate word sequence based on the at least two first candidate words included in the first summary label;

[0352] calculate a second difference term corresponding to each candidate word sequence based on the first prediction probability sequence corresponding to each first candidate word in each candidate word sequence and the corresponding first reference probability sequence;

[0353] calculate the second loss based on the first difference term, including:

[0354] calculate the second loss based on the first difference term and the second difference term.

[0355] Optionally, in an embodiment, the second difference term corresponding to each candidate word sequence is calculated based on the first prediction probability sequence corresponding to each first candidate word in each candidate word sequence and the corresponding first reference probability sequence, including:

[0356] generate a sequence prediction probability distribution based on the first prediction probability sequence corresponding to each first candidate word in each candidate word sequence, and generate a sequence reference probability distribution based on the first reference probability sequence corresponding to each first candidate word in the candidate word sequence;

[0357] calculate the second difference term corresponding to each candidate word sequence according to the sequence prediction probability distribution corresponding to each candidate word sequence and the sequence reference probability distribution.

[0358] Optionally, in an implementation, generating the sequence prediction probability distribution based on the first prediction probability sequence corresponding to each first candidate word in each candidate word sequence comprises:

[0359] Splicing the first prediction probability sequence corresponding to each first candidate word in each candidate word sequence according to the first candidate word arrangement order to obtain a spliced sequence;

[0360] Generating the sequence prediction probability distribution based on the spliced sequence and the number of first candidate words contained in the candidate word sequence.

[0361] Optionally, in an implementation, determining the target neural network model based on the updated preset neural network model comprises:

[0362] Obtaining a second summary label corresponding to the sample text data, the first quality score of the first summary label being higher than the second quality score of the second summary label;

[0363] Inputting the sample text data into the updated preset neural network model to obtain at least one second prediction probability sequence corresponding to at least one first candidate word contained in the first summary label and at least one third prediction probability sequence corresponding to at least one second candidate word contained in the second summary label output by the preset neural network model;

[0364] Calculating a third loss based on the first summary label, the second summary label, the second prediction probability sequence and the third prediction probability sequence;

[0365] Obtaining a second reference probability sequence corresponding to each first candidate word, and calculating a fourth loss based on the second prediction probability sequence corresponding to each first candidate word and the second reference probability sequence, and the third prediction probability sequence corresponding to each second candidate word;

[0366] Calculating a second target loss according to the third loss and the fourth loss, and updating the parameters of the preset neural network model based on the second target loss to determine the target neural network model.

[0367] Optionally, in an implementation, calculating the fourth loss based on the second prediction probability sequence corresponding to each first candidate word and the second reference probability sequence, and the third prediction probability sequence corresponding to each second candidate word comprises:

[0368] Calculating a first candidate word difference item corresponding to the first summary label based on the second prediction probability sequence corresponding to each first candidate word and the second reference probability sequence;

[0369] Calculating a second candidate word difference item corresponding to the second summary label based on the third prediction probability sequence corresponding to each second candidate word;

[0370] The fourth loss is calculated based on the difference between the first and second candidate words.

[0371] Optionally, in one implementation, a fourth loss is calculated based on the difference between the first candidate word and the difference between the second candidate word, including:

[0372] The total difference term is determined based on the difference terms of the first candidate word and the difference terms of the second candidate word.

[0373] The fourth loss is obtained by normalizing the total difference term.

[0374] Optionally, in one implementation, determining the total difference term based on the difference terms of the first candidate word and the second candidate word includes:

[0375] Obtain the first adjustment parameter corresponding to the difference item of the first candidate word and the second adjustment parameter corresponding to the difference item of the second candidate word;

[0376] The total difference term is calculated based on the first adjustment parameter, the second adjustment parameter, the difference term of the first candidate word, and the difference term of the second candidate word.

[0377] Optionally, in one implementation, calculating the first target loss based on the first loss and the second loss includes:

[0378] Obtain the first loss weight corresponding to the first loss and obtain the second loss weight corresponding to the second loss;

[0379] The first target loss is obtained by weighting the first loss and the second loss based on the first loss weight and the second loss weight.

[0380] Optionally, in one implementation, obtaining the second loss weight corresponding to the second loss includes:

[0381] Obtain the weight curve corresponding to the second loss;

[0382] The second loss weight is determined from the weight curve based on the number of iterations in which the parameters of the preset neural network model are updated during the training period.

[0383] Reference Figure 13 , Figure 13 To implement the structural block diagram of a portion of the terminal 140 according to an embodiment of this disclosure, the terminal includes: a radio frequency (RF) circuit 1310, a memory 1315, an input unit 1330, a display unit 1340, a sensor 1350, an audio circuit 1360, a wireless fidelity (WiFi) module 1370, a processor 1380, and a power supply 1390, among other components. Those skilled in the art will understand that... Figure 13The illustrated terminal 140 structure does not constitute a limitation on a mobile phone or a computer, and can include more or fewer components than shown, or combine certain components, or different component arrangements.

[0384] The RF circuit 1310 can be used for receiving and sending signals in the process of information or communication, in particular, receiving the downlink information from the base station and processing by the processor 1380; in addition, sending the uplink data to the base station.

[0385] The memory 1315 can be used for storing software programs and modules, and the processor 1380 executes various functions of the content terminal and text summary generation by running the software programs and modules stored in the memory 1315.

[0386] The input unit 1330 can be used for receiving input digital or character information, and generating key signal input related to the setting and function control of the content terminal. Specifically, the input unit 1330 can include a touch panel 1331 and other input devices 1332.

[0387] The display unit 1340 can be used for displaying input information or provided information and various menus of the content terminal. The display unit 1340 can include a display panel 1341.

[0388] The audio circuit 1360, the speaker 1361, and the microphone 1362 can provide an audio interface.

[0389] In the embodiment, the processor 1380 included in the object terminal 140 can execute the text summary generation method of the previous embodiment.

[0390] The embodiments of the present application can be applied to various scenarios, including but not limited to intelligent text generation, text recommendation, etc.

[0391] Figure 14A structure block diagram of a part of the server 110 implementing the embodiments of the present disclosure. The server 110 can be quite different in configuration or performance, and can include one or more central processing units (CPUs) 1422 (e.g., one or more processors) and a memory 1432, one or more storage media 1430 (e.g., one or more mass storage devices) storing applications 1442 or data 1444. Among them, the memory 1432 and the storage medium 1430 can be temporary storage or persistent storage. The programs stored in the storage medium 1430 can include one or more modules (not shown in the figure), each of which can include a series of instruction operations in the server. Further, the central processing unit 1422 can be configured to communicate with the storage medium 1430 and execute the series of instruction operations in the storage medium 1430 on the server.

[0392] The server 110 can also include one or more power supplies 1426, one or more wired or wireless network interfaces 1450, one or more input / output interfaces 1458, and / or one or more operating systems 1441, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0393] The central processing unit 1422 in the server 110 can be used to execute the text summary generation method of the embodiments of the present disclosure.

[0394] The embodiments of the present disclosure also provide a computer-readable storage medium for storing a computer program, the computer program being used to execute the text summary generation method of each of the above embodiments.

[0395] The embodiments of the present disclosure also provide a computer program product including a computer program. The processor of the electronic device reads the computer program and executes it, so that the electronic device executes the text summary generation method as described above.

[0396] The terms "first", "second", "third", "fourth" and the like in the description of the present disclosure and the above drawings, if any, are used to distinguish similar objects, and do not necessarily indicate a particular order or sequence. It should be understood that the data thus used can be interchanged, where appropriate, so that the embodiments of the present disclosure described herein can be implemented in other sequences than those illustrated or described herein. In addition, the terms "comprise" and "include" and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product or apparatus that includes a list of steps or units as an example is not necessarily limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to such processes, methods, products or apparatuses.

[0397] It should be understood that in the present disclosure, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the relationship between the associated contents, which means that there can be three relationships, for example, "A and / or B" can represent three cases: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated contents. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0398] It should be understood that in the description of the embodiments of the present disclosure, the meaning of multiple (or multiple items) is two or more, greater than, less than, more than, etc. are not included in the number, above, below, etc. are understood to include the number.

[0399] In the embodiments of the present disclosure, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined target, and can be implemented entirely or partially by using software, hardware (such as a processing circuit or a memory) or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an overall module or unit that includes the functions of the module or unit.

[0400] In several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the described device embodiments are merely schematic. For example, the division of the units is merely logical function division. There can be another division manner for the actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0401] The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments.

[0402] In addition, each functional unit in the various embodiments of the present disclosure can be integrated in one processing unit, or each unit can exist physically as a separate unit, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware, or in the form of software functional units.

[0403] When the integrated unit is implemented in the form of software functional units and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such an understanding, the technical solutions of the present disclosure essentially, or the part that contributes to the prior art, or all or a part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in the various embodiments of the present disclosure. The foregoing storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and various media that can store program codes.

[0404] It should also be understood that the various embodiments provided by the present disclosure can be combined in any manner to achieve different technical effects.

[0405] The above is a specific explanation of the embodiments of the present disclosure, but the present disclosure is not limited to the above-described embodiments, and those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present disclosure, and these equivalent modifications or substitutions are included in the scope defined by the claims of the present disclosure.

Claims

1. A method for generating a text summary, characterized by, The method comprises: obtaining a target text and a summary generation prompt text; inputting the target text and the summary generation prompt text into a pre-trained target neural network model to obtain a text summary corresponding to the target text; the training process of the target neural network model comprises: obtaining training sample data, the training sample data comprising sample text data and a first summary label corresponding to the sample text data; inputting the sample text data into a preset neural network model to obtain at least one first predicted probability sequence corresponding to at least one first candidate word contained in the first summary label output by the preset neural network model, the first predicted probability sequence comprising output probabilities corresponding to a plurality of reference words in a reference word library, and calculating a first loss based on the first summary label and the at least one first predicted probability sequence; the first candidate word is a word corresponding to each word position in the summary generated by the preset neural network model for the sample text data, and the reference word is a word that can appear for each word position; obtaining a first reference probability sequence corresponding to each first candidate word, and calculating a second loss based on the first predicted probability sequence corresponding to each first candidate word and the first reference probability sequence, wherein the first probability distribution of the first reference probability sequence corresponds to a first balance higher than the second probability distribution of the second predicted probability sequence. According to the first loss and the second loss, a first target loss is calculated, and the parameters of the preset neural network model are updated based on the first target loss, and the target neural network model is determined based on the updated preset neural network model.

2. The method of claim 1, wherein, The method comprises: based on the first predicted probability sequence corresponding to each first candidate word and the first reference probability sequence, a first difference term corresponding to each first candidate word is calculated; the second loss is calculated based on the first difference term.

3. The method of claim 2, wherein, The first summary label contains at least two first candidate words, and before the second loss is calculated based on the first difference term, the method further comprises: determining at least one candidate word sequence based on at least two first candidate words contained in the first summary label; based on the first predicted probability sequence corresponding to each first candidate word in each candidate word sequence and the corresponding first reference probability sequence, a second difference term corresponding to each candidate word sequence is calculated; the second loss is calculated based on the first difference term and the second difference term. The method comprises:

4. The method of claim 3, wherein, based on the first predicted probability sequence corresponding to each first candidate word in each candidate word sequence and the corresponding first reference probability sequence, a second difference term corresponding to each candidate word sequence is calculated; generate a sequence prediction probability distribution based on the first prediction probability sequence corresponding to each first candidate word in each of the candidate word sequences, and generate a sequence reference probability distribution based on the first reference probability sequence corresponding to each first candidate word in the candidate word sequence; calculate a second difference term corresponding to each of the candidate word sequences according to the sequence prediction probability distribution and the sequence reference probability distribution corresponding to each of the candidate word sequences.

5. The method of claim 4, wherein, The generating a sequence prediction probability distribution based on the first prediction probability sequence corresponding to each first candidate word in each of the candidate word sequences comprises: concatenate the first prediction probability sequence corresponding to each first candidate word in each of the candidate word sequences according to a first candidate word arrangement order to obtain a concatenated sequence; generate a sequence prediction probability distribution based on the concatenated sequence and the number of first candidate words contained in the candidate word sequence.

6. The method of claim 1, wherein, The determining the target neural network model based on the updated preset neural network model comprises: obtain a second summary label corresponding to the sample text data, wherein a first quality score of the first summary label is higher than a second quality score of the second summary label; input the sample text data into the updated preset neural network model to obtain at least one second prediction probability sequence corresponding to the at least one first candidate word contained in the first summary label and at least one third prediction probability sequence corresponding to at least one second candidate word contained in the second summary label output by the preset neural network model; calculate a third loss based on the first summary label, the second summary label, the second prediction probability sequence, and the third prediction probability sequence; obtain a second reference probability sequence corresponding to each first candidate word, and calculate a fourth loss based on the second prediction probability sequence corresponding to each first candidate word and the second reference probability sequence, and the third prediction probability sequence corresponding to each second candidate word; calculate a second target loss according to the third loss and the fourth loss, and update parameters of the preset neural network model based on the second target loss to determine a target neural network model.

7. The method of claim 6, wherein, The calculating a fourth loss based on the second prediction probability sequence corresponding to each first candidate word and the second reference probability sequence, and the third prediction probability sequence corresponding to each second candidate word comprises: calculate a first candidate word difference term corresponding to the first summary label based on the second prediction probability sequence corresponding to each first candidate word and the second reference probability sequence; calculate a second candidate word difference term corresponding to the second summary label based on the third prediction probability sequence corresponding to each second candidate word; calculate a fourth loss according to the first candidate word difference term and the second candidate word difference term.

8. The method of claim 7, wherein, The calculating a fourth loss according to the first candidate word difference term and the second candidate word difference term comprises: determine a total difference term based on the first candidate word difference term and the second candidate word difference term; normalize the total difference term to obtain a fourth loss.

9. The method of claim 8, wherein, The determining the total difference item based on the first candidate word difference item and the second candidate word difference item comprises: obtaining a first adjustment parameter corresponding to the first candidate word difference item and a second adjustment parameter corresponding to the second candidate word difference item; calculating a total difference item based on the first adjustment parameter, the second adjustment parameter, the first candidate word difference item and the second candidate word difference item.

10. The method of claim 1, wherein, The calculating the first target loss according to the first loss and the second loss comprises: obtaining a first loss weight corresponding to the first loss and a second loss weight corresponding to the second loss; weighting the first loss and the second loss based on the first loss weight and the second loss weight to obtain the first target loss.

11. The method of claim 10, wherein, The obtaining the second loss weight corresponding to the second loss comprises: obtaining a weight curve corresponding to the second loss; determining the second loss weight from the weight curve based on an iteration round of updating parameters of the preset neural network model in a training period.

12. A text abstract generation apparatus characterized by comprising: comprise: an obtaining unit, configured to obtain a target text and an abstract generation prompt text; a generating unit, configured to input the target text and the abstract generation prompt text into a pre-trained target neural network model to obtain a text abstract corresponding to the target text; a training process of the target neural network model comprises: obtaining training sample data, the training sample data comprising sample text data and a first abstract label corresponding to the sample text data; inputting the sample text data into a preset neural network model to obtain at least one first predicted probability sequence corresponding to at least one first candidate word contained in the first abstract label output by the preset neural network model, the first predicted probability sequence comprising output probabilities corresponding to a plurality of reference words in a reference word library, and calculating a first loss based on the first abstract label and the at least one first predicted probability sequence; the first candidate word is a word corresponding to each word position in an abstract generated by the preset neural network model for the sample text data, and the reference word is a word that can appear for each word position; obtaining a first reference probability sequence corresponding to each first candidate word, and calculating a second loss based on the first predicted probability sequence corresponding to each first candidate word and the first reference probability sequence, a first probability distribution of the first reference probability sequence corresponding to a first balance higher than a second balance corresponding to a second probability distribution of the first predicted probability sequence; calculating a first target loss according to the first loss and the second loss, updating parameters of the preset neural network model based on the first target loss, and determining the target neural network model based on the updated preset neural network model.

13. An electronic device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to implement the text abstract generation method according to any one of claims 1 to 11.

14. A storage medium storing a computer program, characterized in that, The computer program is executed by the processor to implement the text abstract generation method according to any one of claims 1 to 11.

15. A computer program product comprising a computer program which is read and executed by a processor of an electronic device, causing the electronic device to perform the text summarization method according to any one of claims 1 to 11.

Images