Text abstract generation method and device, electronic equipment and storage medium

Abstract

The invention provides a text abstract generation method and device, electronic equipment and a storage medium. The method comprises the steps of obtaining a target text and an abstract to generate a prompt text; inputting the target text and the abstract generation prompt text into a target neural network model to obtain a text abstract; the training process of the target neural network model comprises the steps of obtaining training sample data; inputting the sample text data into a preset neural network model to obtain a first prediction probability sequence corresponding to a first candidate word contained in the first abstract tag, and calculating a first loss based on the first abstract tag and the first prediction probability sequence; obtaining a first reference probability sequence of the first candidate word, and calculating a second loss based on the first prediction probability sequence of the first candidate word and the first reference probability sequence; and updating parameters of a preset neural network model according to the first loss and the second loss, and determining a target neural network model based on the updated preset neural network model. According to the embodiment of the invention, the text abstract generation accuracy can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a text summary generation method, device, electronic device and storage medium. Background Art

[0002] With the development of the film and television industry, the number of film and television scripts is also increasing. Since the amount of text contained in film and television scripts is relatively large, and they include relatively complex plots and character relationships, it is very difficult to quickly understand the content of film and television scripts and evaluate whether the scripts are suitable for film and television adaptation. Therefore, it is necessary to generate accurate script summaries based on film and television scripts.

[0003] In related technologies, film and television scripts are often input into a preset neural network model. The neural network model can extract the required content from the script to generate a text summary so that the subject can quickly understand the script content, greatly improving the reading efficiency of film and television scripts.

[0004] However, the accuracy of text summary generation for film and television scripts in related technologies is low. Summary of the invention

[0005] The embodiments of the present disclosure provide a text summary generation method, device, electronic device and storage medium, which can improve the accuracy of summarizing text content.

[0006] According to one aspect of the present disclosure, a text summary generation method is provided, comprising:

[0007] Obtain target text and summary generation prompt text;

[0008] Inputting the target text and the summary generation prompt text into a pre-trained target neural network model to obtain a text summary corresponding to the target text;

[0009] The training process of the target neural network model includes:

[0010] Acquire training sample data, where the training sample data includes sample text data and a first summary label corresponding to the sample text data;

[0011] Inputting the sample text data into a preset neural network model, obtaining at least one first prediction probability sequence output by the preset neural network model and corresponding to at least one first candidate word included in the first summary label, wherein the first prediction probability sequence includes output probabilities corresponding to a plurality of reference words in a reference word library, and calculating a first loss based on the first summary label and the at least one first prediction probability sequence;

[0012] Obtaining a first reference probability sequence corresponding to each of the first candidate words, and calculating a second loss based on the first predicted probability sequence corresponding to each of the first candidate words and the first reference probability sequence, wherein a first balance corresponding to a first probability distribution of the first reference probability sequence is higher than a second balance corresponding to a second probability distribution of the first predicted probability sequence;

[0013] A first target loss is calculated according to the first loss and the second loss, and the parameters of the preset neural network model are updated based on the first target loss, and a target neural network model is determined based on the updated preset neural network model.

[0014] According to one aspect of the present disclosure, a text summary generating device is provided, comprising:

[0015] An acquisition unit, used for acquiring a target text and a summary generation prompt text;

[0016] A generating unit, configured to input the target text and the summary generating prompt text into a pre-trained target neural network model to obtain a text summary corresponding to the target text;

[0017] The training process of the target neural network model includes:

[0018] Acquire training sample data, where the training sample data includes sample text data and a first summary label corresponding to the sample text data;

[0019] Inputting the sample text data into a preset neural network model, obtaining at least one first prediction probability sequence output by the preset neural network model and corresponding to at least one first candidate word included in the first summary label, wherein the first prediction probability sequence includes output probabilities corresponding to a plurality of reference words in a reference word library, and calculating a first loss based on the first summary label and the at least one first prediction probability sequence;

[0020] Obtaining a first reference probability sequence corresponding to each of the first candidate words, and calculating a second loss based on the first predicted probability sequence corresponding to each of the first candidate words and the first reference probability sequence, wherein a first balance corresponding to a first probability distribution of the first reference probability sequence is higher than a second balance corresponding to a second probability distribution of the first predicted probability sequence;

[0021] A first target loss is calculated according to the first loss and the second loss, and the parameters of the preset neural network model are updated based on the first target loss, and a target neural network model is determined based on the updated preset neural network model.

[0022] Optionally, in one implementation, the calculating the second loss based on the first predicted probability sequence corresponding to each of the first candidate words and the first reference probability sequence includes:

[0023] Calculate a first difference item corresponding to each of the first candidate words based on the first predicted probability sequence corresponding to each of the first candidate words and the first reference probability sequence;

[0024] A second loss is calculated based on the first difference term.

[0025] Optionally, in one implementation, the first summary tag includes at least two first candidate words, and before calculating the second loss based on the first difference item, the method further includes:

[0026] Determine at least one candidate word sequence based on at least two first candidate words included in the first summary tag;

[0027] Calculate a second difference item corresponding to each of the candidate word sequences based on the first predicted probability sequence corresponding to each of the first candidate words in each of the candidate word sequences and the corresponding first reference probability sequence;

[0028] The calculating the second loss based on the first difference term includes:

[0029] A second loss is calculated based on the first difference term and the second difference term.

[0030] Optionally, in one implementation, the calculating the second difference item corresponding to each candidate word sequence based on the first predicted probability sequence corresponding to each first candidate word in each candidate word sequence and the corresponding first reference probability sequence includes:

[0031] Generate a sequence prediction probability distribution based on the first prediction probability sequence corresponding to each of the first candidate words in each of the candidate word sequences, and generate a sequence reference probability distribution based on the first reference probability sequence corresponding to each of the first candidate words in the candidate word sequence;

[0032] The second difference item corresponding to each candidate word sequence is calculated according to the sequence prediction probability distribution corresponding to each candidate word sequence and the sequence reference probability distribution.

[0033] Optionally, in one implementation, generating a sequence prediction probability distribution based on the first prediction probability sequence corresponding to each first candidate word in each candidate word sequence includes:

[0034] splicing the first predicted probability sequences corresponding to each first candidate word in each candidate word sequence according to the arrangement order of the first candidate words to obtain a spliced ​​sequence;

[0035] A sequence prediction probability distribution is generated based on the concatenated sequence and the number of first candidate words included in the candidate word sequence.

[0036] Optionally, in one implementation, determining a target neural network model based on the updated preset neural network model includes:

[0037] Acquire a second summary tag corresponding to the sample text data, wherein a first quality score of the first summary tag is higher than a second quality score of the second summary tag;

[0038] Inputting the sample text data into the updated preset neural network model, obtaining at least one second prediction probability sequence corresponding to the at least one first candidate word included in the first summary label and at least one third prediction probability sequence corresponding to the at least one second candidate word included in the second summary label output by the preset neural network model;

[0039] calculating a third loss based on the first summary label, the second summary label, the second predicted probability sequence, and the third predicted probability sequence;

[0040] Obtain a second reference probability sequence corresponding to each of the first candidate words, and calculate a fourth loss based on the second predicted probability sequence and the second reference probability sequence corresponding to each of the first candidate words, and the third predicted probability sequence corresponding to each of the second candidate words;

[0041] A second target loss is calculated according to the third loss and the fourth loss, and the parameters of the preset neural network model are updated based on the second target loss to determine the target neural network model.

[0042] Optionally, in one implementation, the calculating the fourth loss based on the second predicted probability sequence and the second reference probability sequence corresponding to each of the first candidate words, and the third predicted probability sequence corresponding to each of the second candidate words, includes:

[0043] Calculate a first candidate word difference item corresponding to the first summary label based on the second predicted probability sequence corresponding to each first candidate word and the second reference probability sequence;

[0044] Calculate the second candidate word difference item corresponding to the second summary label based on the third prediction probability sequence corresponding to each of the second candidate words;

[0045] A fourth loss is calculated based on the first candidate word difference item and the second candidate word difference item.

[0046] Optionally, in one implementation, the calculating the fourth loss according to the first candidate word difference item and the second candidate word difference item includes:

[0047] Determining a total difference item based on the first candidate word difference item and the second candidate word difference item;

[0048] The total difference term is normalized to obtain the fourth loss.

[0049] Optionally, in one implementation, determining a total difference item based on the first candidate word difference item and the second candidate word difference item includes:

[0050] Obtaining a first adjustment parameter corresponding to the first candidate word difference item and a second adjustment parameter corresponding to the second candidate word difference item;

[0051] A total difference term is calculated based on the first adjustment parameter, the second adjustment parameter, the first candidate word difference term, and the second candidate word difference term.

[0052] Optionally, in one implementation, calculating a first target loss according to the first loss and the second loss includes:

[0053] Obtaining a first loss weight corresponding to the first loss and obtaining a second loss weight corresponding to the second loss;

[0054] The first loss and the second loss are weightedly calculated based on the first loss weight and the second loss weight to obtain a first target loss.

[0055] Optionally, in one implementation, the obtaining a second loss weight corresponding to the second loss includes:

[0056] Obtaining a weight curve corresponding to the second loss;

[0057] Based on the iterative rounds of updating the parameters of the preset neural network model during the training cycle, a second loss weight is determined from the weight curve.

[0058] In the text summary generation method of the embodiment of the present disclosure, a target text and a summary generation prompt text are obtained; the target text and the summary generation prompt text are input into a pre-trained target neural network model to obtain a text summary corresponding to the target text; the training process of the target neural network model includes: obtaining training sample data, the training sample data includes sample text data and a first summary label corresponding to the sample text data; inputting the sample text data into a preset neural network model to obtain at least one first prediction probability sequence corresponding to at least one first candidate word included in the first summary label output by the preset neural network model, the first prediction probability sequence includes output probabilities corresponding to multiple reference words in a reference word library, and a first loss is calculated based on the first summary label and the at least one first prediction probability sequence; obtaining a first reference probability sequence corresponding to each first candidate word, and calculating a second loss based on the first prediction probability sequence corresponding to each first candidate word and the first reference probability sequence, wherein a first balance corresponding to a first probability distribution of the first reference probability sequence is higher than a second balance corresponding to a second probability distribution of the first prediction probability sequence; calculating a first target loss based on the first loss and the second loss, updating the parameters of the preset neural network model based on the first target loss, and determining a target neural network model based on the updated preset neural network model.

[0059] Thus, when training the target neural network model for generating text summaries, the disclosed embodiment guides the prediction probability sequence predicted by the target neural network model at each word position by setting a reference probability sequence. Since the probability distribution balance of the reference probability sequence is higher than the probability distribution balance of the prediction probability sequence, the overfitting problem caused by the predicted output of the target neural network model at each word position being too close to the label can be avoided. This can improve the accuracy of predicting words at each word position during the text summary generation process, thereby improving the accuracy of text summary generation.

[0060] Other features and advantages of the present disclosure will be described in the following description, and partly become apparent from the description, or understood by practicing the present disclosure. The purpose and other advantages of the present disclosure can be realized and obtained by the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The accompanying drawings are used to provide further understanding of the technical solution of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the technical solution of the present disclosure and do not constitute a limitation on the technical solution of the present disclosure.

[0062] Figure 1 is a system architecture diagram of a method for generating a text summary according to an embodiment of the present disclosure;

[0063] Figure 2A It is a schematic diagram of an embodiment of the present disclosure applied to a scenario of generating a script plot summary;

[0064] Figure 2B is another schematic diagram of an embodiment of the present disclosure applied to a scenario of generating a script plot summary;

[0065] Figure 2C is another schematic diagram of an embodiment of the present disclosure applied to a scenario of generating a script plot summary;

[0066] Figure 3 is a flow chart of a text summary generation method according to an embodiment of the present disclosure;

[0067] Figure 4 is a schematic diagram of inputting text in a text summary generation interface according to an embodiment of the present disclosure;

[0068] Figure 5 is a structural schematic diagram of a target neural network model according to an embodiment of the present disclosure;

[0069] Figure 6 is a flow chart of a target neural network model training process according to an embodiment of the present disclosure;

[0070] Figure 7 is a schematic diagram of a preset neural network model outputting a prediction probability sequence corresponding to a plurality of candidate words according to an embodiment of the present disclosure;

[0071] Figure 8 is a schematic diagram of a weight curve corresponding to a second loss weight according to an embodiment of the present disclosure;

[0072] Fig. 9 is a schematic diagram of a corresponding relationship between a first loss weight and a second loss weight according to an embodiment of the present disclosure;

[0073] Fig.10 is another flowchart of a method for generating a text summary according to an embodiment of the present disclosure;

[0074] Fig.11 is a schematic diagram of training a first target neural network model based on a reference neural network model according to an embodiment of the present disclosure;

[0075] Fig.12 is a structural schematic diagram of a text summary generating device according to an embodiment of the present disclosure;

[0076] Fig.13 is a terminal structure diagram for implementing various methods according to an embodiment of the present disclosure;

[0077] Fig.14 It is a server structure diagram for implementing various methods according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0078] In order to make the purpose, technical solution and advantages of the present disclosure more clear, the present disclosure is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present disclosure and are not used to limit the present disclosure.

[0079] Before further describing the embodiments of the present disclosure in detail, the nouns and terms involved in the embodiments of the present disclosure are described. The nouns and terms involved in the embodiments of the present disclosure are subject to the following interpretations:

[0080] Large Language Model (LLM): refers to a deep learning model trained with a large amount of text data, which can generate natural language text or understand the meaning of language text. LLM can handle a variety of natural language tasks, such as text classification, question answering, dialogue, etc. The core idea of ​​LLM is to learn the patterns and language structure of natural language through large-scale unsupervised training, store a large amount of natural language knowledge, and better understand and generate natural text.

[0081] Neural Networks: Neural Networks is an important technology in machine learning, which is inspired by the structure and function of the human brain. Neural networks are composed of a large number of neuron nodes, which interact with each other through different connections (weights) to form a complex network. Neural networks can automatically learn features in data and handle complex problems through a multi-level network structure.

[0082] In the related art, a large language model can be used to generate a summary of text content. However, for text content in certain fields, since the large language model has not been trained specifically in the field, the accuracy of summary generation is low. In this regard, the model can be fine-tuned and trained using preset sample texts and corresponding summary tags, so that the model can learn the text content in a specific field, thereby improving the accuracy of text summary generation. However, although fine-tuning with labels can improve the accuracy of summary generation for a certain field, due to the limited number of labels, supervised training will cause the output results of the model to be close to the words in the limited labels, thereby causing the model to overfit during the fine-tuning training process. In other words, when the model generates a summary corresponding to the text content, the probability of generating the words contained in the summary label is high, while the probability of generating other words that can more accurately express the text content is low. The similarity of the summaries generated for different text contents is high, and the precise wording for summaries of different text contents cannot be achieved, resulting in low accuracy of text summary generation. In this regard, the embodiment of the present disclosure provides a text summary generation method, in order to improve the accuracy of text summary generation.

[0083] System architecture and scenario description of the application of the embodiments of the present disclosure

[0084] Figure 1 It is a system architecture diagram applied by the text summary generation method according to the embodiment of the present disclosure, which includes a terminal 140, the Internet 130, a gateway 120, a server 110, and the like.

[0085] The terminal 140 includes various forms such as desktop computers, laptop computers, PDAs (personal digital assistants), mobile phones, vehicle-mounted terminals, smart home appliances, and dedicated terminals. In addition, it can be a single device or a collection of multiple devices. For example, multiple devices are connected through a local area network and work together using a common display device to form a terminal 140. The terminal 140 can also communicate with the Internet 130 in a wired or wireless manner to exchange data.

[0086] The server 110 refers to a computer system that can provide certain services to the terminal 140. Compared with the ordinary terminal 140, the server 110 has very high requirements in terms of stability, security, performance, etc. The server 110 can be a high-performance computer in a network platform, a cluster of multiple high-performance computers, a part (such as a machine) of a high-performance computer, a combination of parts (such as machines) of multiple high-performance computers, etc.

[0087] The gateway 120 is also called an internetwork connector or a protocol converter. The gateway realizes network interconnection at the transport layer and is a computer system or device that acts as a converter. The gateway is a translator between two systems that use different communication protocols, data formats or languages, or even completely different architectures. At the same time, the gateway can also provide filtering and security functions. The message sent by the terminal 140 to the server 110 must be sent to the corresponding server 110 through the gateway 120. The message sent by the server 110 to the terminal 140 must also be sent to the corresponding terminal 140 through the gateway 120.

[0088] The text summary generation method of the embodiment of the present disclosure may be executed by an electronic device, which may be a terminal 140 or a server 110 , that is, the text summary generation method may be executed by the terminal 140 or the server 110 , or may be executed jointly by the terminal 140 and the server 110 .

[0089] The embodiments of the present disclosure can be applied in various scenarios, such as generating a summary of a relatively complex script. Figure 2A The scene shown summarizes the contents of different scenes in the script; Figure 2B The scenes shown here organize the development of the script's plot; and Figure 2C The scene shown is screening a large number of scripts, etc.

[0090] The script content may involve multiple characters, complex plots, time and location information. The script content may include multiple scenes, and different scenes may be related. Summarizing different scenes based on specific conditions is conducive to quickly organizing the scene content. Figure 2A As shown, the script 211 can be divided into multiple scenes 212; a summary is generated for each scene based on the text summary generation method of the embodiment of the present disclosure to obtain a corresponding scene summary 213; and multiple scene summaries 213 are sorted to obtain multiple scene summaries 214. When it is desired to summarize the plots corresponding to the characters A1 and A2 from the contents of multiple scenes, the character plots can be summarized based on the multiple scene summaries 214 to obtain the plot 215 of the character A1 and the plot 216 of the character A2.

[0091] Another example Figure 2B As shown, the script 221 can be divided into multiple scenes 222; after generating a scene summary 223 corresponding to each scene, the entire script can be sorted out based on the scene summary 223 corresponding to each scene to obtain a plot development 224. Based on the plot development 224, the subject can be assisted to quickly understand the script.

[0092] Another example Figure 2CAs shown, the script 231 can be divided into multiple scenes 232; the text summarization method of the disclosed example can be used to generate a summary of the content of each scene to obtain a scene summary 233, and a role summary can be generated for different characters in each scene based on the content of each scene to obtain a role summary 234. Since the summary content has the ability to summarize and is relatively concise, the content of each scene can be quickly understood through the scene summary 233, and the scene content can be screened to obtain target scenes 235, so as to retain important target scenes and discard insignificant scenes; the content of each character can be quickly understood through the character summary 234, and the characters can be screened to obtain target characters 236, so as to retain important characters for filming and discard irrelevant characters.

[0093] It can be seen that when faced with more complex script content, the text summary generation method of the embodiment of the present disclosure can generate an accurate summary so that the text content can be quickly understood based on the summary.

[0094] General description of the disclosed embodiments

[0095] According to an embodiment of the present disclosure, a text summary generation method is provided. The text summary generation method provided by the embodiment of the present disclosure can be applied to scenarios such as script content summary generation. Figure 3 As shown, a flowchart of a text summary generation method provided by the present disclosure is provided. The text summary generation method may include:

[0096] Step 310: Obtain the target text and summary generation prompt text.

[0097] The target text can be the text for which summary generation is required. The target text can be the text content entered by the object through the text input box, or the content uploaded in the form of a file. The target text can be obtained by obtaining the text content entered by the object in the text input box as the target text, or by parsing the text file uploaded by the object to obtain the target text.

[0098] Summary generation prompt text can be text that provides guidance for the summary generation task of the model. The summary generation prompt text can include requirements for text summary generation so that the model can generate a text summary based on the summary generation requirements. For example, in a script summary generation task, the summary generation prompt text is: "Give a plot summary based on the given script text, and describe the events, time, place, cause, process and result in the script in 80 words or less." Through this summary generation prompt text, the model can generate a plot summary based on the script text, and put forward summary generation requirements to the model, requiring the generated summary to contain the above specified content, and specifying the word limit of the summary.

[0099] The summary generation prompt text can be pre-set based on the summary generation scenario. Different summary generation prompt texts can be pre-set for different summary generation scenarios. For example, for a script summary generation task, a corresponding summary generation prompt text can be pre-set; for a news summary generation task, a corresponding summary generation prompt text can also be pre-set. After obtaining the target text, the corresponding summary generation prompt text can be automatically selected based on the type of the target text, or the object can select the summary generation prompt text for executing the summary generation task from multiple preset summary generation prompt texts according to the summary generation requirements.

[0100] The summary generation prompt text may also be a prompt text entered by the subject in the text input box. The prompt text entered by the subject in the text input box is obtained as the summary generation prompt text. The text entered by the subject through the text input box may include the summary generation prompt text and the target text, for example Figure 4 As shown, the subject inputs in the text input box 411 in the text summary generation interface 410: "Give a plot summary based on the given script text, and require a description of the events, time, place, cause, process and results that occurred in the script, within 80 words. The script text is: XXX." The input text is uploaded by triggering the summary generation control 412 for summary generation. In the text input by the subject, "Give a plot summary based on the given script text, and require a description of the events, time, place, cause, process and results that occurred in the script, within 80 words" is the summary generation prompt text, and "XXX" is the target text. After obtaining the text input by the subject through the text summary generation interface 410, the text can be recognized to parse the summary generation prompt text and the target text.

[0101] Step 320: Input the target text and summary generation prompt text into the pre-trained target neural network model to obtain a text summary corresponding to the target text.

[0102] The target neural network model is a neural network model used to generate a summary of the target text based on the summary generation prompt text. The target neural network model can be a large language model with language recognition and generation capabilities.

[0103] In one embodiment, the model structure of the target neural network model can be as follows: Figure 5As shown, the target neural network model 500 includes a text encoding layer 510, a position encoding layer 520, an encoding network 530 and a linear transformation layer 540. The text encoding layer 510 can be used to encode the input text into a fixed-length context vector, which captures the contextual semantic information in the input text. The position encoding layer 520 can be used to add position information to each position of the input text to help the model understand the semantic and contextual analysis between each word in the text. There can be multiple encoding networks 530, and each encoding network 530 can be a transformer stacking structure for capturing contextual information in the input text through a self-attention mechanism. The self-attention mechanism allows the model to consider all elements in the sequence when processing each element, so as to better understand the semantic information in the input text. The linear transformation layer 540 can convert the feature vector output by the encoding network into a probability distribution on the vocabulary through a linear transformation, and convert the vector into a readable text output.

[0104] The target neural network model can be pre-trained. In one embodiment, Figure 6 As shown, the training process of the target neural network model includes:

[0105] Step 610: Obtain training sample data.

[0106] The training sample data can be applied to the training of the target neural network model, including the sample text data and the first summary label corresponding to the sample text data.

[0107] The sample text data may include sample prompt text and sample text to be generated. The sample prompt text is similar to the summary generation prompt text in the above embodiment, and the sample text to be generated is similar to the target text in the above embodiment, which will not be described in detail here.

[0108] The first summary tag may be a result of summarizing the sample text data. In one embodiment, the first summary tag may be manually generated based on the sample text data. That is, the first summary tag is obtained by manually summarizing the sample text data.

[0109] In another embodiment, the first summary label can be generated based on an existing large language model. The sample text data is input into the large language model, and based on the guidance of the sample prompt text in the sample text data, the large language model can extract a summary based on the sample text to be generated to obtain the first summary label. Using the large language model to generate the first summary label corresponding to the sample text data can improve the generation efficiency of the first summary label, thereby improving the efficiency of obtaining training sample data.

[0110] The accuracy of summaries generated by existing large language models is low. For example, for scripts with more complex content, the generated summary may contain factual errors or character errors. Therefore, the summary generated by the large language model based on sample text data can be corrected to obtain a more accurate summary generation result, which can be used as the first summary label. Data correction can be done manually or using an open source large language model.

[0111] After obtaining the first summary label, the sample text data and the first summary label may be stored in correspondence to obtain training sample data.

[0112] Step 620: Input the sample text data into a preset neural network model, obtain at least one first prediction probability sequence output by the preset neural network model and corresponding to at least one first candidate word included in the first summary label, and calculate a first loss based on the first summary label and the at least one first prediction probability sequence.

[0113] The preset neural network model can be an open source neural network model that has been trained using text samples in a large number of common scenarios. Training the preset neural network model as a target neural network model can be supervised fine-tuning of the preset neural network model based on the scenario of summary generation. The target neural network model is obtained by updating and fine-tuning the parameters of the preset neural network model. Therefore, the model architecture of the preset neural network model is the same as the model architecture of the target neural network model in the aforementioned implementation method, which will not be repeated here.

[0114] After obtaining the preset neural network model, the sample text data can be input into the preset neural network model. The preset neural network model can output at least one first prediction probability sequence corresponding to at least one first candidate word included in the first summary tag based on the sample text data. The first prediction probability sequence includes output probabilities corresponding to multiple reference words in the reference word library.

[0115] The preset neural network model can output multiple prediction probability sequences corresponding to multiple candidate words. The candidate words can be the words corresponding to each character position in the summary generated by the preset neural network model for the sample text data. For example, the maximum number of characters in the summary generated by the preset neural network model is 4096, so there are 4096 candidate words, and the preset neural network model can output the prediction probability sequence corresponding to the 4096 candidate words. The prediction probability sequence corresponding to each candidate word may include the output probability of multiple reference words in the reference vocabulary at this character position. The reference word can be a word that may appear in each character position. For example, there are 6400 reference words in the reference vocabulary, so the prediction probability sequence corresponding to each candidate word can be a 6400-dimensional vector, which contains the probability that each reference word may appear in this character position. For example Figure 7As shown, at word position 0001, among the multiple reference words corresponding to the candidate words, the probability of reference word w1 appearing is 0.1, the probability of reference word w2 appearing is 0.13, the probability of reference word w3 appearing is 0.21, the probability of reference word w4 appearing is 0.06, and the probability of reference word w6400 appearing is 0.17. It can be seen that the preset neural network model can output a two-dimensional array with a row dimension equal to 4096 and a column dimension equal to 6400, each row represents a candidate word, and each column represents a reference word.

[0116] At least one first candidate word included in the first summary tag may be a candidate word corresponding to each character position in the first summary tag. For example, if the first summary tag contains 15 characters, there are 15 first candidate words in total. The first prediction probability sequence corresponding to the first candidate word may be a prediction probability sequence corresponding to the character position of the first candidate word among multiple prediction probability sequences output by the preset neural network model. Figure 7 Taking the multiple prediction probability sequences shown as an example, when the first summary label contains 15 characters, the first prediction probability sequence can be the prediction probability sequence corresponding to candidate words 0001 to 0015. Therefore, at least one first prediction probability sequence corresponding to at least one first candidate word can be obtained from the multiple prediction probability sequences corresponding to the multiple candidate words output by the preset neural network based on the number of first candidate words in the first summary label.

[0117] After obtaining at least one first prediction probability sequence, a first loss may be calculated based on the first summary label and the at least one first prediction probability sequence. The first loss may measure the difference between the output of the preset neural network model and the first summary label. The smaller the first loss, the higher the summary accuracy of the output of the preset neural network model.

[0118] In one embodiment, calculating a first loss based on a first summary label and at least one first prediction probability sequence includes:

[0119] For each first candidate word corresponding to the first summary label, calculating a word difference item corresponding to the first candidate word based on the candidate word label corresponding to the first candidate word in the first summary label and the corresponding first prediction probability sequence;

[0120] A first loss is calculated based on at least one word difference item corresponding to at least one first candidate word.

[0121] The candidate word label can be the true value corresponding to the first candidate word in the first summary label. For example, if the first summary label is "plot content", then "drama", "plot", "content" and "content" are respectively the candidate word labels of the first candidate word corresponding to each character position in the first summary label. The word difference item corresponding to the first candidate word can represent the difference between the predicted probability of the preset neural network model for multiple reference words corresponding to the first candidate word and the true value corresponding to the first candidate word in the first summary label. The word difference item corresponding to the first candidate word can be obtained by calculating the cross entropy between the candidate word label corresponding to the first candidate word and the first predicted probability sequence. The calculation formula of the cross entropy can be expressed as Formula 1:

[0122]

[0123] In formula 1, l i y may represent the word difference item corresponding to the i-th first candidate word in the first summary tag; m may represent the number of reference words, and c may represent the sequence number of the reference word. ic It can indicate whether the cth reference word is equal to the candidate word label corresponding to the first candidate word. When the cth reference word is equal to the candidate word label corresponding to the first candidate word, y ic can be equal to 1; otherwise, when the cth reference word is not equal to the candidate word label corresponding to the first candidate word, y ic Can be equal to 0. ic It can be the prediction probability corresponding to the cth reference word in the first prediction probability sequence. For example, the candidate word label corresponding to the first candidate word in the first summary label is w2, and the multiple reference words are: w1, w2, w3 and w4, and their corresponding first prediction probability sequence is [0.1, 0.3, 0.2, 0.4]. Therefore, based on Formula 1, the word difference term corresponding to the first candidate word is equal to -(0*log0.1+1*log0.3+0*log0.2+0*log0.4)=0.522.

[0124] After calculating the word difference item corresponding to each first candidate word, the first loss may be calculated based on at least one word difference item corresponding to at least one first candidate word.

[0125] In one implementation, the first loss may be equal to the average of at least one word difference term, which may be expressed as Formula 2:

[0126]

[0127] In Formula 2, L1 can represent the first loss, n can represent the number of first candidate words, and l iIt can represent the word difference item corresponding to the i-th first candidate word. For example, there are 5 first candidate words, and the corresponding word difference items are: 0.522, 0.615, 0.369, 0.537 and 0.414 respectively. The first loss calculated based on formula 2 is equal to (0.522+0.615+0.369+0.537+0.414) / 5=0.4914.

[0128] Measuring the difference between the output of the preset neural network model and the first summary label based on the prediction difference of each first candidate word can ensure that the trained target neural network model can obtain a relatively accurate generation result for each first candidate word, thereby improving the accuracy of the overall summary generation.

[0129] Step 630: Obtain a first reference probability sequence corresponding to each first candidate word, and calculate a second loss based on the first predicted probability sequence corresponding to each first candidate word and the first reference probability sequence.

[0130] The first reference probability sequence may include reference probabilities of multiple reference words corresponding to the first candidate word. The first balance corresponding to the first probability distribution of the first reference probability sequence is higher than the second balance corresponding to the second probability distribution of the first prediction probability sequence. Balance can be used to indicate the balance of the probability distribution of multiple reference words. The closer the probabilities are, the higher the balance. Therefore, the reference probabilities of multiple reference words in the first reference probability sequence are relatively close, and the first balance of the first probability distribution is relatively high, which can be used to balance the second probability distribution of multiple reference words in the first prediction probability sequence corresponding to the first candidate word, avoiding the problem of too high prediction probability of words in the first summary label and too low probability of other reference words, and overfitting of the model training, and maintaining the diversity of the preset neural network model that can generate different summary words for different texts.

[0131] In one embodiment, the first reference probability sequence may be a preset sequence of multiple reference words with the same reference probability. For example, the reference probability of each reference word in the first reference probability sequence is 1. The first reference probability sequence obtained in this way can ensure that the probability distribution corresponding to each reference word has a high balance.

[0132] In another embodiment, obtaining a first reference probability sequence corresponding to each first candidate word includes:

[0133] Obtain a reference neural network model;

[0134] The sample text data is input into the reference neural network model to obtain a first reference probability sequence corresponding to each first candidate word.

[0135] In one embodiment, the reference neural network model may be a preset neural network model that has a relatively balanced distribution of reference word prediction probabilities in other text generation tasks. The neural network model may have the same model structure as the preset neural network model. Acquiring the preset neural network model as the reference neural network model may improve the efficiency of acquiring the reference neural network model.

[0136] In another embodiment, the reference neural network model may be a model corresponding to a preset neural network model and obtained by training in an iteration round before the current iteration round.

[0137] It takes multiple rounds of iterations to train the preset neural network model as the target neural network model. With the update iteration of the model parameters, the prediction accuracy of the preset neural network model for multiple reference words is improved, the prediction probability for the words in the first summary tag will increase, and the prediction probability of other reference words will decrease. Therefore, as the iteration rounds advance, the balance of the first prediction probability sequence output by the preset neural network model will gradually decrease. Therefore, the model obtained by the iteration round training before the current iteration round corresponding to the preset neural network model can be used as a reference neural network model. For example, the model obtained in the previous iteration round corresponding to the preset neural network model currently being trained can be used as a reference neural network model. For example, the current training round is i, and the model used in the i-th round of training is the model obtained in the i-1th round of iteration. The model obtained in the previous iteration round of the i-1th round, that is, the i-2th round of iteration, can be obtained as a reference neural network model.

[0138] Using the model obtained by training in the iteration round before the current iteration round and corresponding to the preset neural network model as the reference neural network model can balance the first probability distribution of the first prediction probability sequence while maintaining the prediction accuracy.

[0139] After obtaining the reference neural network model, the sample text data can be input into the reference neural network model to obtain a first reference probability sequence corresponding to each first candidate word. Obtaining the first reference probability sequence corresponding to each first candidate word based on the reference neural network model can ensure that the first reference probability sequence and the first predicted probability sequence are generated based on the same model structure, thereby achieving the goal of obtaining a first reference probability sequence with a higher degree of balance while maintaining the accuracy of the first probability distribution prediction of multiple reference words in the first reference probability sequence.

[0140] After obtaining the first reference probability sequence corresponding to each first candidate word, the second loss can be calculated based on the first predicted probability sequence corresponding to each first candidate word and the first reference probability sequence. The second loss can be used to measure the balance gap between the first predicted probability sequence and the first reference probability sequence, so as to bring the first balance of the first probability distribution of the first predicted probability sequence closer to the second balance of the second probability distribution of the first reference probability sequence.

[0141] In one embodiment, calculating the second loss based on the first predicted probability sequence and the first reference probability sequence corresponding to each first candidate word includes:

[0142] Calculate a first difference item corresponding to each first candidate word based on a first predicted probability sequence corresponding to each first candidate word and a first reference probability sequence;

[0143] The second loss is calculated based on the first difference term.

[0144] The first difference term may indicate a gap between a second probability distribution of multiple reference words output by a preset neural network model for each first candidate word and a first probability distribution corresponding to the first reference probability sequence.

[0145] In one implementation, the process of calculating the first difference item corresponding to the first candidate word based on the first predicted probability sequence and the first reference probability sequence can be expressed as Formula 3:

[0146]

[0147] In formula 3, y w Can represent the first summary tag; F i Can represent the first summary label y w The first difference item corresponding to the i-th first candidate word in ; x may represent sample text data input into the preset neural network model; It can represent the first predicted probability sequence corresponding to the i-th first candidate word obtained based on the sample text data x, It can represent the first reference probability sequence corresponding to the i-th first candidate word. It can be seen that the first difference term can determine the gap between the first probability distribution and the second probability distribution by measuring the amount of additional information generated when the first probability distribution corresponding to the first reference probability sequence approximates the second probability distribution corresponding to the first predicted probability sequence.

[0148] Based on Formula 3, the first difference term between the first predicted probability sequence and the first reference probability sequence can be determined by calculating the probability difference between each reference word in the first predicted probability sequence and the probability difference in the first reference probability sequence, which can be specifically expressed as Formula 4:

[0149]

[0150] In formula 4, It can represent the probability value of the jth reference word in the first prediction probability sequence corresponding to the i-th first candidate word, It can represent the probability value of the jth reference word in the first reference probability sequence corresponding to the i-th first candidate word. The difference item of the j-th reference word corresponding to the ith first candidate word can be calculated, and the first difference item corresponding to the ith first candidate word can be obtained by adding the difference items corresponding to multiple reference words.

[0151] For example, the reference words include w1, w2, w3 and w4, the first prediction probability sequence corresponding to the i-th first candidate word is [0.1, 0.5, 0.2, 0.2], and the corresponding first reference probability sequence is [0.25, 0.25, 0.25, 0.25]. Based on Formula 4, it can be calculated that the first difference item corresponding to the first candidate word is -(0.1*log(0.1 / 0.25)+0.5*log(0.5 / 0.25)+0.2*log(0.2 / 0.25)+0.2*log(0.2 / 0.25))=0.072.

[0152] After calculating the first difference item corresponding to each first candidate word, the second loss can be calculated based on at least one first difference item corresponding to at least one first candidate word. In one embodiment, the second loss can be obtained by adding at least one first difference item corresponding to at least one first candidate word.

[0153] Calculating the second loss based on the first difference item corresponding to each first candidate word can ensure that the first probability distribution corresponding to each candidate word can achieve a relatively balanced state, thereby improving the comprehensiveness of predictive balance training for the preset neural network model.

[0154] In another embodiment, the first summary tag includes at least two first candidate words. Based on this, before calculating the second loss based on the first difference item, the method further includes:

[0155] Determine at least one candidate word sequence based on at least two first candidate words included in the first summary tag;

[0156] The second difference item corresponding to each candidate word sequence is calculated based on the first prediction probability sequence corresponding to each first candidate word in each candidate word sequence and the corresponding first reference probability sequence.

[0157] The candidate word sequence may be a sequence consisting of at least two adjacent first candidate words in the first summary tag. For example, if the first summary tag is "plot content", when the candidate word sequence is a sequence consisting of two first candidate words, the candidate word sequence may include "plot", "plot content" and "content".

[0158] When the preset neural network model outputs a sentence, in addition to determining a single word, it also considers the sequence collocation of words or short sentences composed of adjacent words. Some sequences have a higher generation probability, while some sequences have a lower generation probability. After learning the first summary label, the preset neural network model may have an overfitting problem in which the generation probability of one or some word sequences is higher, while the generation probability of other word sequences is lower. Therefore, in order to maintain the diversity of the word sequences generated by the preset neural network model, the word sequence generation of the preset neural network model can also be trained for balance.

[0159] The second difference item corresponding to the candidate word sequence can be a balance difference between the predicted probability distribution of the candidate word sequence output by the preset neural network model and the reference probability distribution corresponding to the candidate word sequence. The balance of the reference probability distribution corresponding to the candidate word sequence is higher than the balance of the predicted probability distribution of the candidate word sequence output by the preset neural network model.

[0160] The balance difference between the predicted probability distribution corresponding to the candidate word sequence and the reference probability distribution can be calculated based on the first predicted probability sequence and the first reference probability sequence corresponding to each candidate word in the candidate word sequence.

[0161] In one embodiment, calculating the second difference item corresponding to each candidate word sequence based on the first predicted probability sequence corresponding to each first candidate word in each candidate word sequence and the corresponding first reference probability sequence includes:

[0162] Generate a sequence prediction probability distribution based on a first prediction probability sequence corresponding to each first candidate word in each candidate word sequence, and generate a sequence reference probability distribution based on a first reference probability sequence corresponding to each first candidate word in the candidate word sequence;

[0163] The second difference item corresponding to each candidate word sequence is calculated according to the sequence prediction probability distribution corresponding to each candidate word sequence and the sequence reference probability distribution.

[0164] The sequence prediction probability distribution may be a prediction probability distribution corresponding to a candidate word sequence, which is used to represent the probability distribution of a plurality of reference words generated by a preset neural network model for each first candidate word in the candidate word sequence. The sequence prediction probability distribution may be generated based on a first prediction probability sequence corresponding to each first candidate word in the candidate word sequence.

[0165] In one embodiment, generating a sequence prediction probability distribution based on a first prediction probability sequence corresponding to each first candidate word in each candidate word sequence includes:

[0166] splicing the first prediction probability sequences corresponding to each first candidate word in each candidate word sequence according to the arrangement order of the first candidate words to obtain a spliced ​​sequence;

[0167] Generate a sequence prediction probability distribution based on the number of first candidate words included in the concatenated sequence and the candidate word sequence.

[0168] First, the first prediction probability sequence corresponding to each first candidate word in the candidate word sequence can be spliced ​​according to the arrangement order of the first candidate words. For example, the candidate word sequence includes two first candidate words, the first prediction probability sequence corresponding to the first first candidate word is [k1, k2, k3, k4], and the first prediction probability sequence corresponding to the second first candidate word is [k5, k6, k7, k8]. The spliced ​​sequence obtained by splicing the first prediction probability sequences is [k1, k2, k3, k4, k5, k6, k7, k8].

[0169] After obtaining the concatenated sequence, a sequence prediction probability distribution can be generated based on the number of first candidate words contained in the concatenated sequence and the candidate word sequence. Since the first prediction probability sequence is a probability distribution of multiple reference words corresponding to the first candidate words generated by a preset neural network model, the sum of the probability values ​​in the first prediction probability sequence is 1. The sum of the probabilities in the concatenated sequence obtained by concatenating the first prediction probability sequences corresponding to at least two first candidate words is equal to the number of first candidate words. In order to more accurately calculate the probability distribution of multiple reference words corresponding to each first candidate word in the candidate word sequence, the probability distribution in the concatenated sequence can be normalized using the number of first candidate words to obtain a sequence prediction probability distribution with a sum of probability values ​​of 1.

[0170] In one implementation, each probability value in the concatenated sequence may be divided by the number of first candidate words, as shown in Formula 5:

[0171] S = concat(t1, t2, ..., tn) / n (Formula 5).

[0172] In Formula 5, S represents the sequence prediction probability distribution corresponding to the candidate word sequence; n represents the number of first candidate words contained in the candidate word sequence; t1, t2, ..., tn represent the first prediction probability sequence corresponding to each first candidate word in the candidate word sequence; concat represents the operation of concatenating the first prediction probability sequence. For example, the candidate word sequence contains two first candidate words, the first prediction probability sequence of the first first candidate word is [0.1, 0.3, 0.2, 0.4], and the first prediction probability sequence of the second first candidate word is [0.3, 0.1, 0.1, 0.5]. Based on Formula 5, the sequence prediction probability distribution corresponding to the candidate word sequence can be obtained as [0.1, 0.3, 0.2, 0.4, 0.3, 0.1, 0.1, 0.5] / 2 = [0.05, 0.15, 0.1, 0.2, 0.15, 0.05, 0.05, 0.25].

[0173] The first prediction probability sequences corresponding to at least two first candidate words are concatenated, and a sequence prediction probability distribution is generated based on the concatenated sequence and the number of first candidate words. The probability distribution of multiple reference words corresponding to each first candidate word in the candidate word sequence can be obtained from the sequence prediction probability distribution. The second difference item corresponding to the entire candidate word sequence is calculated based on the probability distribution of multiple reference words corresponding to each first candidate word, so that the probability of generating a combination of multiple reference words corresponding to at least two first candidate words is balanced, and the accuracy of the balanced training of the word sequence of the preset neural network model is improved.

[0174] Similarly, the sequence reference probability distribution can be a reference probability distribution corresponding to a candidate word sequence, which is used to represent the balanced probability distribution of multiple reference words corresponding to each first candidate word in the candidate word sequence. The sequence reference probability distribution can be generated based on the first reference probability sequence corresponding to each first candidate word in the candidate word sequence. The generation process can be the same as the process of generating a sequence prediction probability distribution based on the first prediction probability sequence corresponding to each first candidate word in each candidate word sequence in the aforementioned embodiment, which will not be repeated here.

[0175] After generating the sequence prediction probability distribution and the sequence reference probability distribution corresponding to each candidate word sequence, the corresponding second difference item can be calculated based on the sequence prediction probability distribution and the sequence reference probability distribution. The calculation method of calculating the second difference item based on the sequence prediction probability distribution and the sequence reference probability distribution can be the same as the calculation method of calculating the first difference item corresponding to the first candidate word based on the first prediction probability sequence corresponding to the first candidate word and the first reference probability sequence in the aforementioned embodiment, and the gap between the sequence prediction probability distribution and the sequence reference probability distribution is determined by measuring the additional information generated when the sequence reference probability distribution approximates the sequence prediction probability distribution.

[0176] The following takes the candidate word sequence composed of two first candidate words as an example to explain in detail the calculation process of the second difference item corresponding to the candidate word sequence. The process of calculating the second difference item for the candidate word sequence composed of two first candidate words can be expressed as Formula 6:

[0177]

[0178] In Formula 6, y w Can represent the first summary tag; F (i,i+1) Can represent the first summary label y w The second difference item corresponding to the candidate word sequence formed by the i-th first candidate word and the i+1-th first candidate word; x can represent the sample text data input into the preset neural network model. It can represent the first prediction probability sequence corresponding to the i-th first candidate word obtained based on the sample text data x; It can represent the first predicted probability sequence corresponding to the i+1th first candidate word. Therefore, It can represent the sequence prediction probability distribution corresponding to the candidate word sequence generated based on the first prediction probability sequence corresponding to the i-th first candidate word and the i+1-th first candidate word. Similarly, It can represent the sequence reference probability distribution corresponding to the candidate word sequence generated based on the first reference probability sequence corresponding to the i-th first candidate word and the i+1-th first candidate word.

[0179] The calculation method of the first difference item described in Formula 3 is the same as that of the second difference item corresponding to the candidate word sequence composed of two first candidate words, which is calculated based on Formula 6 and will not be repeated here.

[0180] Based on the first prediction probability sequence corresponding to each first candidate word in the candidate word sequence, a sequence prediction probability distribution is generated, and based on the first reference probability sequence corresponding to each first candidate word in the candidate word sequence, a sequence reference probability distribution is generated. The balance between the sequence prediction probability distribution and the sequence reference probability distribution corresponding to the candidate word sequence can be determined from the dimension of the first candidate word. The second difference item is calculated based on the sequence prediction probability distribution and the sequence reference probability distribution to ensure that the generation probability distribution of multiple reference words corresponding to each first candidate word in the candidate word sequence can maintain balance, thereby improving the accuracy of the balanced training of the word sequence of the preset neural network model.

[0181] Based on the above implementation, calculating the second loss based on the first difference item includes: calculating the second loss based on the first difference item and the second difference item.

[0182] In one implementation, calculating the second loss based on the first difference term and the second difference term includes:

[0183] Adding a plurality of first difference items corresponding to a plurality of first candidate words to obtain a candidate word difference item;

[0184] Adding at least one second difference item corresponding to at least one candidate word sequence to obtain a candidate word sequence difference item;

[0185] The second loss is obtained by adding the candidate word difference term and the candidate word sequence difference term.

[0186] When the first difference item corresponding to the first candidate word is calculated using the above formula 3, the calculation formula of the candidate word difference item can be expressed as formula 7:

[0187]

[0188] In Formula 7, F seq1 (y w ) can represent the first summary label y w The candidate word difference items corresponding to the multiple first candidate words in the first summary tag are summed up to obtain the candidate word difference items.

[0189] Taking the candidate word sequence composed of two first candidate words as an example, when the second difference item corresponding to the candidate word sequence is calculated using the above formula 6, the calculation formula of the candidate word sequence difference item can be expressed as formula 8:

[0190]

[0192] In Formula 8, F seq2 (y w ) can represent the first summary label y w The candidate word sequence difference item corresponding to at least one candidate word sequence in the first summary tag, n can represent the number of first candidate words in the first summary tag. Each candidate word sequence is composed of two adjacent first candidate words, and n-1 candidate word sequences can be obtained. The second difference items corresponding to the n-1 candidate word sequences are added together to obtain the candidate word sequence difference item.

[0193] The second loss can be obtained by adding the candidate word difference item and the candidate word sequence difference item. The second loss can be expressed as Formula 9:

[0194] L2=F seq1 (y w )+F seq2 (y w )(Formula 9).

[0195] Calculating the second loss based on the first difference item corresponding to each first candidate word and the second difference item corresponding to each candidate word sequence can achieve balanced output of the preset neural network model in both single word dimension and word sequence dimension, thereby avoiding a high output probability of a certain word and a certain word sequence in the first summary label, further maintaining the diversity of summary generation of the preset neural network model for different texts, and improving the accuracy of summary generation.

[0196] Step 640: Calculate a first target loss based on the first loss and the second loss, update the parameters of the preset neural network model based on the first target loss, and determine the target neural network model based on the updated preset neural network model.

[0197] The first loss may be a loss used to measure the accuracy of the output of the preset neural network model, and the second loss may be a loss used to measure the balance of the output of the preset neural network model. Therefore, updating the parameters of the preset neural network model based on the first target loss can maintain the accuracy of the output of the preset neural network model while maintaining the diversity of the output of the preset neural network model.

[0198] In one embodiment, the first target loss can be calculated based on the first loss and the second loss by directly adding the first loss and the second loss to obtain the first target loss. For example, if the first loss is 1.5 and the second loss is 1, then the first target loss obtained is 1+1.5=2.5. Calculating the first target loss by directly adding can ensure that the first loss and the second loss have the same impact on the training of the preset neural network model, thereby improving the fairness of the model accuracy training and the balance training.

[0199] In another embodiment, calculating the first target loss according to the first loss and the second loss includes:

[0200] Obtaining a first loss weight corresponding to the first loss and obtaining a second loss weight corresponding to the second loss;

[0201] The first loss and the second loss are weightedly calculated based on the first loss weight and the second loss weight to obtain the first target loss.

[0202] The first loss weight and the second loss weight can be used to limit the influence of the first loss and the second loss on the training of the preset neural network model. In one embodiment, the first loss weight and the second loss weight can be predetermined values, which can be set according to the degree of accuracy training and balance training of the preset neural network model in actual application. For example, the first loss weight is 0.7 and the second loss weight is 0.3.

[0203] In another implementation, the first loss weight is a predetermined value, and obtaining the second loss weight corresponding to the second loss includes:

[0204] Get the weight curve corresponding to the second loss;

[0205] Based on the iterative rounds of updating the parameters of the preset neural network model during the training cycle, a second loss weight is determined from the weight curve.

[0206] The weight curve may be a curve showing the change of the second loss weight as the model is updated and iterated during the training cycle. In one embodiment, the weight curve may be a sin function waveform curve. As the model is updated and iterated, the second loss weight may be updated according to the change of the weight curve.

[0207] For example Figure 8 As shown in the figure, the maximum value of the second loss weight in the weight curve is 1 and the minimum value is 0. During the training cycle, the model update iteration number is 1000. In the first iteration round, the second loss weight is 0.5; then it gradually increases to the maximum value of 1; then it gradually decreases, and after decreasing to 0, it gradually increases again; until the 500th iteration, the second loss weight returns to 0.5. From the first iteration round to the 500th iteration round, the second loss weight undergoes a sin function cycle (0 degrees-360 degrees). In the 501st iteration round to the 1000th iteration round, the second loss weight changes in the same way.

[0208] Therefore, in each round of iterative training, the second loss weight can be determined from the weight curve based on the iteration round. For example, the weight curve is as follows Figure 8 As shown, when the iteration round is 500, the second loss weight is 0.5.

[0209] The first loss weight may be a predetermined fixed value. The relationship between the first loss weight and the second loss weight may be expressed as: Fig. 9 .like Fig. 9 As shown, the second loss weight changes according to a predetermined weight curve as the number of iterations increases, while the first loss weight adopts a fixed value of 0.3 in each iteration.

[0210] The reason why a fixed value is used for the first loss weight and the second loss weight is updated according to the weight curve is that when training the preset neural network model, setting the first loss weight and the second loss weight based on human experience may cause an imbalance between the accuracy training and the balance training of the model. When the first loss weight is too large and the second loss weight is too small, insufficient training of the balance of the model may cause overfitting of the model. When the first loss weight is too small and the second loss weight is too large, the model output result may be too balanced, affecting the accuracy of the model output. The purpose of training the preset neural network model using the first loss and the second loss is to improve the diversity and balance of the model output while maintaining the accuracy of the model output, thereby improving the accuracy of the model in generating summaries for different texts. Therefore, when determining the first loss weight and the second loss weight, the first loss weight can be set to a fixed value to maintain the accuracy of the model output, and the second loss weight can be changed based on the weight curve, so as to determine the optimal second loss weight that can maintain the balance between the accuracy and balance of the model training.

[0211] Determining the second loss weight based on the weight curve facilitates the evaluation of the impact of different second weight losses on the accuracy training and balance training of the preset neural network model, avoiding problems such as the preset neural network model being too balanced and resulting in a decrease in accuracy, or too high accuracy and resulting in overfitting, and improves the balance between accuracy training and balance training of the preset neural network model.

[0212] After obtaining the first loss weight and the second loss weight, the first loss and the second loss can be weighted based on the first loss weight and the second loss weight to obtain the first target loss. For example, if the first loss weight is 0.7, the second loss weight is 0.3, the first loss is 1.5, and the second loss is 1, then the first target loss is 0.7*1.5+0.3*1=1.35.

[0213] The first loss and the second loss are weightedly calculated based on the first loss weight and the second loss weight to obtain the first target loss. The degree of accuracy training and balance training of the preset neural network model can be adjusted according to actual training needs, thereby improving the flexibility of the preset neural network model training.

[0214] After the first target loss is calculated, the parameters of the preset neural network model can be updated based on the first target loss. When updating the parameters of the preset neural network model, the first target loss can be passed back to the preset neural network model, the gradient of each parameter can be calculated, and the parameters can be updated in the direction of gradient descent. After K rounds of iterations of a training cycle, the target neural network model is obtained.

[0215] In another embodiment, determining a target neural network model based on an updated preset neural network model includes:

[0216] Obtain a second summary tag corresponding to the sample text data;

[0217] Inputting the sample text data into the updated preset neural network model, obtaining at least one second prediction probability sequence corresponding to at least one first candidate word included in the first summary label and at least one third prediction probability sequence corresponding to at least one second candidate word included in the second summary label output by the preset neural network model;

[0218] calculating a third loss based on the first summary label, the second summary label, the second predicted probability sequence, and the third predicted probability sequence;

[0219] Obtaining a second reference probability sequence corresponding to each first candidate word, and calculating a fourth loss based on the second predicted probability sequence and the second reference probability sequence corresponding to each first candidate word, and the third predicted probability sequence corresponding to each second candidate word;

[0220] A second target loss is calculated according to the third loss and the fourth loss, and the parameters of the preset neural network model are updated based on the second target loss to obtain the target neural network model.

[0221] The second summary tag may be another summary different from the first summary tag and obtained based on the sample text data. The second summary tag may be obtained in the same manner as the first summary tag. In one embodiment, the second summary tag may be artificially generated based on the sample text data. In another embodiment, the second summary tag may be generated based on an existing large language model. When generating the second summary tag based on the large language model, the sample prompt text in the sample text data may be different from the sample prompt text when generating the first summary tag, thereby guiding the large language model to generate a summary different from the first summary tag.

[0222] The first quality score of the first summary label is higher than the second quality score of the second summary label. The quality score can be used to measure whether the corresponding summary is accurate and whether it meets the reading preference. When determining the first summary label and the second summary label, two summary labels can be generated manually or by a large language model, and the quality scores of the two summary labels can be evaluated. The summary label with a higher quality score is determined as the first summary label, and the summary label with a lower quality score is determined as the second summary label. The quality score evaluation of the summary labels can be performed by manual comparison to determine which summary label has higher quality; the quality scores of the summary labels can also be evaluated by a large language model.

[0223] The sample text data, the first summary label and the second summary label together constitute training sample data for further training of the preset neural network model.

[0224] The sample text data is input again into the preset neural network model trained by the aforementioned steps 610-640, and at least one second prediction probability sequence corresponding to at least one first candidate word included in the first summary label and at least one third prediction probability sequence corresponding to at least one second candidate word included in the second summary label output by the preset neural network model are obtained. The second candidate word indicates the candidate word corresponding to each character position in the second summary label. The second prediction probability sequence and the third prediction probability sequence are similar to the first prediction probability sequence in the aforementioned embodiment, and the second prediction probability sequence includes the prediction probabilities of multiple reference words corresponding to the first candidate word output by the updated preset neural network model, and the third prediction probability sequence includes the prediction probabilities of multiple reference words corresponding to the second candidate word.

[0225] After obtaining the second prediction probability sequence and the third prediction probability sequence, the third loss may be calculated based on the first summary label, the second summary label, the second prediction probability sequence, and the third prediction probability sequence.

[0226] In one embodiment, calculating the third loss based on the first summary label, the second summary label, the second prediction probability sequence, and the third prediction probability sequence includes:

[0227] Determining a first generation probability of a first summary label generated by a preset neural network model based on at least one second prediction probability sequence;

[0228] Determining a second generation probability of a second summary label generated by a preset neural network model based on at least one third prediction probability sequence;

[0229] A third loss is calculated based on the first generation probability and the second generation probability.

[0230] The second prediction probability sequence indicates the generation probability distribution of multiple reference words corresponding to the first candidate word in the first summary tag. Therefore, in one embodiment, determining the first generation probability of the first summary tag generated by the preset neural network model based on at least one second prediction probability sequence includes:

[0231] Determine a candidate word label generation probability based on a second prediction probability sequence corresponding to each first candidate word and a candidate word label corresponding to the first summary label;

[0232] A first generation probability is calculated based on at least one candidate word label generation probability corresponding to at least one first candidate word.

[0233] The candidate word label can be the true value corresponding to the first candidate word in the first summary label, which has been described in detail in the previous embodiment and will not be repeated here. The second predicted probability sequence contains the probability corresponding to the candidate word label. Therefore, the probability corresponding to the candidate word label can be directly obtained from the second predicted probability sequence as the candidate word label generation probability. For example, the candidate word label corresponding to a first candidate word in the first summary label is w1, and the multiple reference words are: w1, w2, w3 and w4 in sequence. The second predicted probability sequence corresponding to the first candidate word is [0.3, 0.2, 0.1, 0.4]. Therefore, the candidate word label generation probability corresponding to the candidate word label w1 is 0.3.

[0234] Based on the calculation of the first generation probability of at least one candidate word label generation probability corresponding to at least one first candidate word, the at least one candidate word label generation probability can be multiplied to obtain the first generation probability corresponding to the first summary label. For example, the first summary label is "plot content", and based on the second prediction probability sequence corresponding to the four first candidate words, it can be determined that the preset neural network model generates candidate word label generation probabilities of "drama", "plot", "content" and "content" in these four word positions: 0.3, 0.5, 0.2 and 0.3 respectively. Therefore, the first generation probability of the first summary label is 0.3*0.5*0.2*0.3=0.009.

[0235] The process of determining the second generation probability of the preset neural network model generating the second summary label based on at least one third prediction probability sequence is the same as the process of determining the first generation probability, which is not repeated here.

[0236] After determining the first generation probability and the second generation probability, the third loss may be calculated based on the first generation probability and the second generation probability. In one embodiment, the process of calculating the third loss based on the first generation probability and the second generation probability may be expressed as Formula 10:

[0237]

[0238] In formula 10, It can represent the training sample data composed of x, y1 and y2, x can represent the sample text data, y1 can represent the first summary label, and y2 can represent the second summary label. P(y1|x) can represent the first generation probability of generating the first summary label based on the sample text data; P(y2|x) can represent the second generation probability of generating the second summary label based on the sample text data. σ can represent a logical function, which is used to normalize the difference between the first generation probability and the second generation probability, and then map the normalized result to a smaller range based on the log function to obtain the third loss L3.

[0239] When training the preset neural network model based on the third loss obtained from formula 10, in order to reduce the third loss, the training goal can be: to increase the distance between the first generation probability and the second generation probability, that is, to increase the first generation probability of the first summary label generated by the preset neural network model after parameter update, and reduce the second generation probability of the second summary label, so that the target neural network model can generate a text summary with a higher quality score.

[0240] In another embodiment, before calculating the third loss based on the first generation probability and the second generation probability, the method further includes: obtaining a first generation reference probability corresponding to the first summary tag and a second generation reference probability corresponding to the second summary tag. Therefore, calculating the third loss based on the first generation probability and the second generation probability includes: calculating the third loss based on the first generation probability, the second generation probability, the first generation reference probability, and the second generation reference probability.

[0241] The first generation reference probability can be the probability that a model with relatively accurate summary generation generates a first summary label; similarly, the second generation reference probability can be the probability that the model generates a second summary label. The above-mentioned model with high summary generation accuracy can be the preset neural network model obtained based on the first target loss update parameter in the aforementioned step 640. Therefore, the sample text data can be input into the preset neural network model obtained in step 640 to obtain the first generation reference probability and the second generation reference probability.

[0242] For a model that generates summary more accurately, the first generation reference probability of generating a first summary label with a relatively high quality score will be greater than the second generation reference probability of generating a second summary label. In order to allow the model to further improve the probability of generating the first summary label and reduce the probability of generating the second summary label on this basis, the parameters of the preset neural network model can be updated to further improve the first generation probability on the basis of the first generation reference probability.

[0243] In one implementation, the process of calculating the third loss based on the first generation probability, the second generation probability, the first generation reference probability, and the second generation reference probability can be expressed as Formula 11:

[0244]

[0245] In Formula 11, Q(y1|x) may represent the first generation reference probability of generating the first summary label y1 based on the sample text data x; Q(y2|x) may represent the second generation reference probability of generating the second summary label y2 based on the sample text data x. β is a parameter that can be used to control the degree of deviation between the first generation probability and the first generation reference probability, and between the second generation probability and the second generation reference probability. By setting β, the influence of the first generation reference probability and the second generation reference probability on the first generation probability and the second generation probability can be controlled. The meanings of other parameters are the same as those in Formula 10 and are not repeated here.

[0246] When the preset neural network model is trained based on the third loss obtained by formula 11, the training goal can be: increase the difference between the first generation probability and the first generation reference probability, and reduce the difference between the second generation probability and the second generation reference probability, so that The value of becomes larger, making The value of becomes smaller, the distance between the two values ​​is increased, and the third loss is reduced. In this way, the probability of the preset neural network model tending to generate the first summary label is increased, and the accuracy of the summary generation of the preset neural network model is improved.

[0247] Calculating the third loss based on the first generation probability, the second generation probability, the first generation reference probability and the second generation reference probability can allow the preset neural network model to further improve the accuracy of summary generation on the basis of a model with relatively accurate summary generation.

[0248] After calculating the third loss, the second reference probability sequence corresponding to each first candidate word can be obtained, and the fourth loss can be calculated based on the second predicted probability sequence and the second reference probability sequence corresponding to each first candidate word, and the third predicted probability sequence corresponding to each second candidate word.

[0249] The second reference probability sequence corresponding to the first candidate word is similar to the first reference probability sequence in the aforementioned step, except that the output probabilities corresponding to the reference words in the second reference probability sequence are different for different training scenarios. Therefore, the balance of the second reference probability sequence is higher than that of the second prediction probability sequence. The method for obtaining the second reference probability sequence is also the same as that for the first reference probability sequence, which will not be described in detail here.

[0250] The fourth loss can be used to measure the difference between the second predicted probability sequence and the second reference probability sequence when outputting the first summary label, and the predicted probability distribution corresponding to each second candidate word when outputting the second summary label. The third loss can increase the probability of the preset neural network model generating a first summary label with a higher quality score. Therefore, based on the fourth loss, the preset neural network model is trained for balance, which can avoid the overfitting problem that the target neural network model outputs too high a probability for the words in the first summary label and too low a probability for the words in the second summary label, thereby maintaining the balance of the probability distribution of multiple reference words generated by the preset neural network.

[0251] In one embodiment, calculating the fourth loss based on the second predicted probability sequence and the second reference probability sequence corresponding to each first candidate word, and the third predicted probability sequence corresponding to each second candidate word, includes:

[0252] Calculate the first candidate word difference item corresponding to the first summary label based on the second prediction probability sequence corresponding to each first candidate word and the second reference probability sequence;

[0253] Calculate the second candidate word difference item corresponding to the second summary label based on the third prediction probability sequence corresponding to each second candidate word;

[0254] The fourth loss is calculated based on the difference item of the first candidate word and the difference item of the second candidate word.

[0255] The first candidate word difference item corresponding to the first summary label can be calculated based on the second predicted probability sequence corresponding to each first candidate word and the second reference probability sequence. The specific calculation process can be the same as the process of calculating the second loss based on the first predicted probability sequence corresponding to each first candidate word and the first reference probability sequence in the aforementioned step 630. In one embodiment, the first difference item corresponding to each first candidate word can be calculated by calculating the second predicted probability sequence corresponding to each first candidate word and the second reference probability sequence, and the first candidate word difference item corresponding to the first summary label can be calculated based on the first difference item. In another embodiment, the first summary label includes at least two first candidate words, and at least one candidate word sequence can be determined based on the at least two first candidate words; the second difference item corresponding to each candidate word sequence is calculated based on the second predicted probability sequence corresponding to each first candidate word in each candidate word sequence and the second reference probability sequence, and the first candidate word difference item corresponding to the first summary label is calculated based on the first difference item and the second difference item. The detailed process has been described in detail in the aforementioned embodiment and will not be repeated here.

[0256] The second candidate word difference item corresponding to the second summary label can be calculated based on the third prediction probability sequence corresponding to each second candidate word. The second candidate word difference item can be calculated based on the difference item corresponding to each second candidate word.

[0257] In one implementation, the calculation process of the difference item corresponding to each second candidate word can be expressed as Formula 12:

[0258]

[0259] In Formula 12, G i It can represent the difference item corresponding to the i-th second candidate word in the second summary tag; It can be represented by generating a second summary label y based on the sample text data x based on the preset neural network model 2 The third predicted probability sequence corresponding to the i-th second candidate word in . Therefore, the difference term corresponding to the second candidate word is equal to the sum of the information entropy corresponding to each reference word in the third predicted probability sequence. When the output probabilities corresponding to all reference words are equal, the probability distribution of the third predicted probability sequence is uniform, and the difference term corresponding to the second candidate word reaches the maximum value.

[0260] The process of calculating the difference item of the second candidate word based on the difference item corresponding to each second candidate word can be expressed as Formula 13:

[0261]

[0262] In Formula 13, F en (y 2 ) can represent the second summary label y 2 The corresponding second candidate word difference item; n can represent the number of second candidate words. The difference item corresponding to each second candidate word is summed up to obtain the second candidate word difference item corresponding to the second summary label.

[0263] After obtaining the first candidate word difference item and the second candidate word difference item, a fourth loss may be calculated based on the first candidate word difference item and the second candidate word difference item.

[0264] In one implementation, calculating the fourth loss according to the difference item of the first candidate word and the difference item of the second candidate word includes:

[0265] Determining a total difference term based on the first candidate word difference term and the second candidate word difference term;

[0266] The total difference term is normalized to obtain the fourth loss.

[0267] In one implementation, the total difference term may be obtained by summing the first candidate word difference term and the second candidate word difference term.

[0268] In another embodiment, determining a total difference term based on the first candidate word difference term and the second candidate word difference term includes:

[0269] Obtaining a first adjustment parameter corresponding to the first candidate word difference item and a second adjustment parameter corresponding to the second candidate word difference item;

[0270] A total difference term is calculated based on the first adjustment parameter, the second adjustment parameter, the first candidate word difference term and the second candidate word difference term.

[0271] Based on the first adjustment parameter and the second adjustment parameter, the influence of the first summary label and the second summary label on the preset neural network model training can be set. When the first summary label and the second summary label are required to have the same degree of influence on the preset neural network model training, the first adjustment parameter and the second adjustment parameter can be set to the same value, such as 0.5. The first adjustment parameter and the second adjustment parameter can be pre-set based on the requirements in actual training or training experience.

[0272] After determining the first adjustment parameter and the second adjustment parameter, the total difference item can be calculated based on the first adjustment parameter, the second adjustment parameter, the first candidate word difference item and the second candidate word difference item, which can be specifically expressed as Formula 14:

[0273] T=β 1 *F seq (y 1 )+β2*F en (y 2 )(Formula 14).

[0274] In Formula 14, T can represent the total difference term; β 1 can represent the first adjustment parameter; β2 can represent the second adjustment parameter; F seq (y 1 ) can represent the first summary label y 1 The first candidate word difference item obtained; F en (y 2 ) can represent the second summary label y 2 The obtained second candidate word difference item.

[0275] Based on the first adjustment parameter and the second adjustment parameter, a weighted calculation is performed on the first candidate word difference item and the second candidate word difference item to obtain a total difference item. The influence of the first summary label and the second summary label on the preset neural network model training can be adjusted according to the training requirements in practical applications, thereby improving the flexibility of the preset neural network model training.

[0276] After calculating the total difference term, the total difference term can be normalized to obtain the fourth loss. Specifically, the calculation process of the fourth loss can be expressed as Formula 15:

[0277]

[0278] In formula 15, It can represent the training sample data composed of x, y1 and y2, x can represent the sample text data, y1 can represent the first summary label, and y2 can represent the second summary label. T can represent the total difference term, σ can represent the logic function, which is used to normalize the total difference term, and then map the normalized result to a smaller range based on the log function to obtain the fourth loss L4.

[0279] In one of the aforementioned implementations, the first difference item corresponding to each first candidate word can be calculated based on the second predicted probability sequence corresponding to each first candidate word and the second reference probability sequence; the second difference item corresponding to each candidate word sequence is calculated based on the second predicted probability sequence corresponding to each first candidate word in each candidate word sequence and the second reference probability sequence, and the first candidate word difference item corresponding to the first summary label is calculated based on the first difference item and the second difference item. In another implementation, the total difference item is calculated based on the first adjustment parameter and the second adjustment parameter. Therefore, in one implementation, the calculation process of the fourth loss can be expressed as Formula 16:

[0280]

[0281] In Formula 16, F seq1 (y 1 ) may represent the sum of multiple first difference items corresponding to multiple first candidate words in the first summary tag, F seq2 (y 1 ) may represent the sum of at least one second difference item corresponding to at least one candidate word sequence in the first summary tag, and the sum of multiple first difference items and the sum of at least one second difference item constitute the first candidate word difference item; based on the first adjustment parameter β 1 , the second adjustment parameter β2, the first candidate word difference item and the second candidate word difference item F en (y 2 ), we can get the total difference term.

[0282] Calculating the fourth loss based on the first candidate word difference item corresponding to each first candidate word and the second candidate word difference item corresponding to each second candidate word realizes balanced training for the first summary label and the second summary label respectively, thereby improving the accuracy of balanced training for the preset neural network model.

[0283] After the fourth loss is calculated, the second target loss can be calculated according to the third loss and the fourth loss. The second target loss can be calculated in the same manner as the process of calculating the first target loss according to the first loss and the second loss in step 640, which will not be repeated here.

[0284] After calculating the first loss, the parameters of the preset neural network model can be updated based on the first target loss. In one training cycle, the target neural network model can be obtained based on the parameter update of the predetermined number of iterations.

[0285] Further training the preset neural network model based on the first summary label and the second summary label can achieve intensive training of the preset neural network model. By comparing the quality of the first summary label and the second summary label, the accuracy of summary generation by the preset neural network model is further improved. At the same time, during the intensive training process, a second reference probability sequence with higher balance is used to maintain the diversity of summary generation of the preset neural network model, avoiding the problem of overfitting of the first summary label by the preset neural network model after intensive training. Therefore, through this implementation, the summary of the target neural network model obtained by training can output more accurate summaries for different texts.

[0286] In summary, the text summary generation method of the embodiment of the present disclosure uses the first loss calculated based on the first label and at least one first prediction probability sequence corresponding to at least one first candidate word when training the target neural network model for generating text summaries, so that the target neural network model can obtain a more accurate result when generating text summaries. In addition, the second loss is calculated based on the first prediction probability sequence corresponding to each first candidate word and the reference probability sequence. Since the first balance of the first probability distribution of the reference probability sequence is higher than the second balance of the second probability distribution of the first prediction probability sequence, the target neural network model can also have a higher balance when generating the first prediction probability sequence corresponding to each first candidate word based on the second loss. In this way, the output probability of some reference words will not be too high and the output probability of other reference words will not be too low due to the guidance of the first label on the generated results. Therefore, the trained target neural network model can avoid overfitting the first training sample data while ensuring the accuracy of text summary generation, maintain the diversity of text summary generation, achieve accurate wording for the text summary corresponding to the target text, and improve the accuracy of text summary generation.

[0287] The embodiments of the present disclosure are described in detail in combination with specific application scenarios.

[0288] like Fig.10 As shown, it is a specific flow chart of the text summary generation method provided by the present disclosure applied to the scenario of script summary generation, and the text summary generation method includes:

[0289] Step 1001: Obtain sample plot data and multiple sample prompt texts, and input each sample prompt text and sample plot data into a large language model to obtain candidate summary tags.

[0290] The sample prompt text can be a text that provides guidance for the model's script summary generation task. The summary generation prompt text can contain the requirements for plot summary generation, so that the model can generate a plot summary based on the summary generation requirements. For example, the sample prompt text is: "Give a plot summary based on the given script text, and describe the events, time, place, cause, process and result of the script in 80 words or less."

[0291] The sample plot data is input into the large language model. Based on the guidance of the sample prompt text, the large language model can extract a summary based on the sample plot data to obtain a candidate summary tag.

[0292] Since the accuracy of summaries generated by existing large language models is low, data correction can be performed on the summaries generated by the large language models to obtain more accurate candidate summary labels. Data correction can be done manually or using open source large language models.

[0293] Different sample prompt texts can generate different plot summaries for the same sample plot data. For example, another sample prompt text is: "Please give a plot summary of the following script text, describing what happened in the script in 80 words or less." Therefore, based on multiple sample prompt texts, multiple candidate summary labels corresponding to the sample plot data can be obtained.

[0294] Step 1002: Determine a first summary tag and a second summary tag from candidate summary tags.

[0295] The first quality score of the first summary tag is greater than the second quality score of the second summary tag. The quality score can be used to measure whether the corresponding summary is accurate and whether it meets the reading preference. When determining the first summary tag and the second summary tag, the quality scores of multiple candidate summary tags can be determined by manual comparison or a large language model, and the first summary tag and the second summary tag can be determined based on the quality scores.

[0296] After the first summary label and the second summary label are determined, the sample plot data and the first summary label may be stored as first training data, and the sample plot data, the first summary label, and the second summary label may be stored as second training data.

[0297] Step 1003: Input the sample plot data in the first training data into a preset neural network model to obtain a plurality of first prediction probability sequences corresponding to a plurality of first candidate words included in the first summary tag.

[0298] The preset neural network model may be an open source neural network model that has been trained using a large number of text samples in common scenarios. Training the preset neural network model as a target neural network model may be supervised fine-tuning of the preset neural network model based on a scenario of script summary generation. The preset neural network model may include a text encoding layer, a position encoding layer, an encoding network, and a linear transformation layer. The text encoding layer may be used to encode the input text into a fixed-length context vector that captures contextual semantic information in the input text. The position encoding layer may be used to add position information to each position of the input text to help the model understand the semantic and contextual analysis between words in the text. The position encoding layer may input a 6400-dimensional vector and output a 4096-dimensional vector. The encoding network may be a transformer stacking structure that is used to capture contextual information in the input text through a self-attention mechanism. The encoding network receives a 4096-dimensional vector and outputs a 6400-dimensional vector. The linear transformation layer may convert the feature vector output by the encoding network into a probability distribution on a vocabulary through a linear transformation, and convert the vector into a readable text output.

[0299] The sample text data is input into a preset neural network model. The preset neural network model can output multiple first prediction probability sequences corresponding to multiple first candidate words included in the first summary label based on the sample plot data. The first prediction probability sequence includes output probabilities corresponding to multiple reference words in the reference vocabulary. The first candidate word can indicate the word corresponding to each character position in the first summary label. The reference word can indicate the word that may be generated by each character position. Therefore, the first prediction probability sequence indicates the probability distribution of the word that may be generated by each character position in the first summary label.

[0300] Step 1004: Calculate a first loss based on the first summary label and a plurality of first prediction probability sequences.

[0301] When calculating the first loss based on the first summary label and multiple first prediction probability sequences, the word difference item corresponding to the first candidate word can be calculated for each first candidate word in the first summary label based on the candidate word label corresponding to the first candidate word in the first summary label and the corresponding first prediction probability sequence.

[0302] The candidate word label may be the true value corresponding to the first candidate word in the first summary label. Based on the probability distribution in the first prediction probability sequence and the cross entropy between the candidate words, the word difference item corresponding to the first candidate word may be calculated, as shown in the above formula 1.

[0303] After calculating the word difference item corresponding to each first candidate word, an average value of multiple word difference items corresponding to multiple first candidate words may be calculated to determine the first loss.

[0304] Step 1005: Obtain a first reference probability sequence corresponding to each first candidate word, and calculate a second loss based on the first predicted probability sequence corresponding to each first candidate word and the first reference probability sequence.

[0305] The first reference probability sequence may include reference probabilities of multiple reference words corresponding to the first candidate word. The first balance corresponding to the first probability distribution of the first reference probability sequence is higher than the second balance corresponding to the second probability distribution of the first prediction probability sequence. The reference probabilities of multiple reference words in the first reference probability sequence are relatively similar, and the first balance of the first probability distribution is relatively high, which can be used to balance the second probability distribution of multiple reference words in the first prediction probability sequence corresponding to the first candidate word.

[0306] The first reference probability sequence can be generated by a reference neural network model based on sample plot data. The reference neural network model can be a preset neural network model that has a relatively balanced distribution of reference word prediction probabilities in other text generation tasks; or it can be a model obtained by training in the iteration round before the current iteration round corresponding to the preset neural network model. The first reference probability sequence can also be a preset probability sequence, for example, the reference probability of each reference word in the first reference probability sequence is 1.

[0307] When calculating the second loss based on the first predicted probability sequence corresponding to each first candidate word and the first reference probability sequence, the first difference item corresponding to each first candidate word can be first calculated based on the first predicted probability sequence corresponding to each first candidate word and the first reference probability sequence. The first difference item can indicate the gap between the second probability distribution of multiple reference words output by the preset neural network model for each first candidate word and the first probability distribution corresponding to the first reference probability sequence. The process of calculating the first difference item can be expressed as Formula 3. By adding the multiple first difference items corresponding to the multiple first candidate words, the candidate word difference item can be obtained, as shown in Formula 7.

[0308] Secondly, at least one candidate word sequence may be determined based on the first candidate words included in the first summary tag. The candidate word sequence may be a sequence formed by two adjacent first candidate words in the first summary tag.

[0309] The second difference item corresponding to each first candidate word sequence is calculated based on the first predicted probability sequence corresponding to each first candidate word in each candidate word sequence and the corresponding first reference probability sequence. The calculation process of the second difference item can be expressed as Formula 6. Multiple second difference items corresponding to multiple candidate word sequences are added together to obtain the candidate word sequence difference item, as shown in Formula 8.

[0310] After obtaining the candidate word difference item and the candidate word sequence difference item, the candidate word difference item and the candidate word sequence difference item may be added to obtain a second loss, as shown in Formula 9.

[0311] Step 1006: Calculate a first target loss according to the first loss and the second loss, and update the parameters of the preset neural network model based on the first target loss.

[0312] Obtain a first loss weight corresponding to the first loss and a second loss weight corresponding to the second loss. The first loss weight may be a predetermined value, and the second loss weight may be determined based on a weight curve and a model iteration round. The weight curve may be a curve showing the change of the second loss weight as the model is updated and iterated during the training cycle, and may be a sin function waveform curve. The second loss weight is determined in the weight curve based on the current iteration round of the preset neural network model in the training cycle.

[0313] The first loss and the second loss are weighted and calculated using the first loss weight and the second loss weight to obtain the first target loss. After the first target loss is calculated, the parameters of the preset neural network model can be updated based on the first target loss. When the parameters of the preset neural network model are updated, the first target loss can be back-propagated to the preset neural network model to calculate the gradients of the parameters and update the parameters in the direction of gradient descent.

[0314] Step 1007: After updating the parameters of the preset neural network model for a first predetermined iterative round, a first target neural network model is obtained, and a first generation reference probability and a second generation reference probability are obtained based on the first target neural network model.

[0315] After a first predetermined iteration round of updates in a training cycle, a first target neural network model is obtained.

[0316] The first generated reference probability may be the probability that the first target neural network model outputs the first summary label; the second generated reference probability may be the probability that the first target neural network model outputs the second summary label. The sample plot data is input into the first target neural network model again to obtain the first generated reference probability and the second generated reference probability. The first generated reference probability and the second generated reference probability can be used in subsequent training processes.

[0317] Step 1008: Input the sample plot data in the second training data into the first target neural network model to obtain multiple second prediction probability sequences corresponding to the multiple first candidate words included in the first summary label, and multiple third prediction probability sequences corresponding to the multiple second candidate words included in the second summary label.

[0318] The first candidate word may indicate the word corresponding to each character position in the first summary tag. The second candidate word may indicate the word corresponding to each character position in the second summary tag. The second prediction probability sequence includes the prediction probabilities of multiple reference words corresponding to the first candidate word output by the updated preset neural network model, and the third prediction probability sequence includes the prediction probabilities of multiple reference words corresponding to the second candidate word.

[0319] When obtaining the second prediction probability sequence, the prediction probability sequence of each word position corresponding to the first summary label can be obtained from the prediction probability sequences corresponding to the multiple word positions output by the first target neural network model. The same is true for obtaining the third prediction probability sequence.

[0320] Step 1009: Calculate a first generation probability based on the first summary label and the second prediction probability sequence, and calculate a second generation probability based on the second summary label and the third prediction probability sequence.

[0321] The first generation probability may be the probability that the first target neural network model of the current iteration round outputs the first summary label; the second generation probability may be the probability that the first target neural network model of the current iteration round outputs the second summary label,

[0322] When calculating the first generation probability, the candidate word label generation probability can be determined based on the second prediction probability sequence corresponding to each first candidate word and the candidate word label corresponding to the first summary label; the candidate word label generation probabilities corresponding to multiple first candidate words are multiplied to obtain the first generation probability corresponding to the first summary label. The calculation process of the second generation probability corresponding to the second summary label is similar.

[0323] Step 1010: Calculate a third loss based on the first generation probability, the second generation probability, the first generation reference probability and the second generation reference probability.

[0324] After obtaining the first generation probability and the second generation probability, the third loss can be calculated based on the first generation probability, the second generation probability, the first generation reference probability and the second generation reference probability. The first generation reference probability and the second generation reference probability are obtained by the first target neural network model trained in step 1007 based on the sample plot data. The calculation process of the third loss can be expressed as the above formula 11.

[0325] Step 1011, obtain the second reference probability sequence corresponding to each first candidate word, and calculate the fourth loss based on the second predicted probability sequence and the second reference probability sequence corresponding to each first candidate word, and the third predicted probability sequence corresponding to each second candidate word.

[0326] The second reference probability sequence may be the same as the first reference probability sequence, or may be different from the first reference probability sequence. The balance of the second reference probability sequence is higher than the balance of the second prediction probability sequence. The second reference probability sequence may be pre-set, or may be generated by a reference neural network model based on sample plot data. The reference neural network model may be a preset neural network model that has a relatively balanced distribution of reference word prediction probabilities in other text generation tasks; it may be a model trained in the iteration round before the current iteration round; or it may be the first target neural network model in step 1007.

[0327] When the reference neural network model is the first target neural network model trained in step 1007, both the third loss and the fourth loss need to rely on the output of the model. Fig.11 As shown, the first target neural network model 1110 of the current iteration round has the same structure as the reference neural network model 1120, and the first target neural network model 1110 of the current iteration round includes a text encoding layer 1111, an embedding layer 1112, multiple encoding networks 1113, and a linear transformation layer 1114. The reference neural network model 1120 includes a text encoding layer 1121, an embedding layer 1122, multiple encoding networks 1123, and a linear transformation layer 1124. The third loss is obtained based on the first generation probability of the output of the first summary label and the second generation probability of the output of the second summary label of the first target neural network model 1110 corresponding to the current iteration round, and the first generation reference probability of the output of the first summary label and the second generation reference probability of the output of the second summary label of the reference neural network model 1120. The fourth loss is calculated based on the second prediction probability sequence corresponding to the first summary label obtained by the encoding network 1113 of the first target neural network model 1110 corresponding to the current iteration round, the third prediction probability sequence corresponding to the second summary label, and the second reference probability sequence corresponding to the first summary label obtained by the encoding network 1123 of the reference neural network model 1120.

[0328] When calculating the fourth loss, the first candidate word difference item corresponding to the first summary label may be calculated based on the second prediction probability sequence corresponding to each first candidate word and the second reference probability sequence. The calculation process of the first candidate word difference item is the same as the calculation process of the second loss in step 1005.

[0329] Then, based on the third prediction probability sequence corresponding to each second candidate word, the second candidate word difference item corresponding to the second summary label is calculated. The calculation process of the second candidate word difference item can be expressed as the above formula 13.

[0330] After obtaining the first candidate word difference item and the second candidate word difference item, the first adjustment parameter corresponding to the first candidate word difference item and the second adjustment parameter corresponding to the second candidate word difference item can be obtained, as shown in the above formula 14, and the total difference item is calculated based on the first adjustment parameter, the second adjustment parameter, the first candidate word difference item and the second candidate word difference item. Finally, the total difference item is normalized to obtain the fourth loss. Therefore, the process of calculating the fourth loss based on the first candidate word difference item and the second candidate word difference item can be expressed as the above formula 16.

[0331] Step 1012: Calculate the second target loss based on the third loss and the fourth loss, and update the parameters of the first target neural network model based on the second target loss. After a second predetermined iterative round of updates, obtain the second target neural network model.

[0332] The second target loss is calculated in the same manner as the first target loss in step 1006. A third loss weight corresponding to the third loss and a fourth loss weight corresponding to the fourth loss may be obtained. The third loss weight may be a predetermined value, and the fourth loss weight may be determined based on a weight curve and a model iteration round. The weight curve may also be a sin function waveform curve. The fourth loss weight is determined in the weight curve based on the current iteration round of the first target neural network model in the training cycle.

[0333] The third loss and the fourth loss are weighted and calculated using the third loss weight and the fourth loss weight to obtain the second target loss. After the second target loss is calculated, the parameters of the first target neural network model can be updated based on the second target loss. When the parameters of the first target neural network model are updated, the second target loss can be back-transferred to the first target neural network model to calculate the gradients of each parameter and update the parameters in the direction of gradient descent. After the second predetermined iteration round of updates in a training cycle, the second target neural network model is obtained.

[0334] Step 1013, obtaining the target script plot and summary generation prompt text; inputting the target script plot and summary generation prompt text into the second target neural network model to obtain the target plot summary.

[0335] The target script plot may be a plot text that needs to be summarized. The summary generation prompt text may be a text that provides guidance for the summary generation task of the model. The target script plot is input into the second target neural network model to generate a target plot summary corresponding to the target script plot.

[0336] Description of the apparatus and device of the present disclosure

[0337] It is to be understood that, although the steps in the above-mentioned flowcharts are sequentially displayed according to the characterization of arrows, these steps are not necessarily executed in sequence according to the order of arrow characterization. Unless there is a clear description in the present embodiment, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the above-mentioned flowcharts can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the steps or stages in other steps.

[0338] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to the characteristics of the target content such as target content attribute information or attribute information sets, the permission or consent of the target content will be obtained first, and the collection, use and processing of these data will comply with relevant laws, regulations and standards. In addition, when the embodiment of the present application needs to obtain the attribute information of the target content, it will obtain the separate permission or separate consent of the target content through a pop-up window or jump to a confirmation page. After clearly obtaining the separate permission or separate consent of the target content, the necessary target content-related data used to enable the normal operation of the embodiment of the present application will be obtained.

[0339] Fig.12 The schematic diagram of the structure of the text summary generation device 1200 provided in the embodiment of the present disclosure. The text summary generation device 1200 includes:

[0340] An acquisition unit 1210 is used to acquire a target text and a summary generation prompt text;

[0341] A generating unit 1220 is used to input the target text and the summary generation prompt text into a pre-trained target neural network model to obtain a text summary corresponding to the target text;

[0342] The training process of the target neural network model includes:

[0343] Acquire training sample data, where the training sample data includes sample text data and a first summary label corresponding to the sample text data;

[0344] Inputting the sample text data into a preset neural network model, obtaining at least one first prediction probability sequence corresponding to at least one first candidate word included in the first summary label output by the preset neural network model, the first prediction probability sequence including output probabilities corresponding to a plurality of reference words in a reference word library, and calculating a first loss based on the first summary label and the at least one first prediction probability sequence;

[0345] Obtaining a first reference probability sequence corresponding to each first candidate word, and calculating a second loss based on the first predicted probability sequence corresponding to each first candidate word and the first reference probability sequence, wherein a first balance corresponding to the first probability distribution of the first reference probability sequence is higher than a second balance corresponding to the second probability distribution of the first predicted probability sequence;

[0346] A first target loss is calculated according to the first loss and the second loss, and the parameters of the preset neural network model are updated based on the first target loss, and a target neural network model is determined based on the updated preset neural network model.

[0347] Optionally, in one implementation, calculating the second loss based on the first predicted probability sequence and the first reference probability sequence corresponding to each first candidate word includes:

[0348] Calculate a first difference item corresponding to each first candidate word based on a first predicted probability sequence corresponding to each first candidate word and a first reference probability sequence;

[0349] The second loss is calculated based on the first difference term.

[0350] Optionally, in one implementation, the first summary tag includes at least two first candidate words, and before calculating the second loss based on the first difference item, the method further includes:

[0351] Determine at least one candidate word sequence based on at least two first candidate words included in the first summary tag;

[0352] Calculate a second difference item corresponding to each candidate word sequence based on a first prediction probability sequence corresponding to each first candidate word in each candidate word sequence and a corresponding first reference probability sequence;

[0353] The second loss is calculated based on the first difference term, including:

[0354] A second loss is calculated based on the first difference term and the second difference term.

[0355] Optionally, in one embodiment, calculating the second difference item corresponding to each candidate word sequence based on the first predicted probability sequence corresponding to each first candidate word in each candidate word sequence and the corresponding first reference probability sequence includes:

[0356] Generate a sequence prediction probability distribution based on a first prediction probability sequence corresponding to each first candidate word in each candidate word sequence, and generate a sequence reference probability distribution based on a first reference probability sequence corresponding to each first candidate word in the candidate word sequence;

[0357] The second difference item corresponding to each candidate word sequence is calculated according to the sequence prediction probability distribution corresponding to each candidate word sequence and the sequence reference probability distribution.

[0358] Optionally, in one embodiment, generating a sequence prediction probability distribution based on a first prediction probability sequence corresponding to each first candidate word in each candidate word sequence includes:

[0359] splicing the first prediction probability sequences corresponding to each first candidate word in each candidate word sequence according to the arrangement order of the first candidate words to obtain a spliced ​​sequence;

[0360] Generate a sequence prediction probability distribution based on the number of first candidate words included in the concatenated sequence and the candidate word sequence.

[0361] Optionally, in one implementation, determining a target neural network model based on an updated preset neural network model includes:

[0362] Obtaining a second summary label corresponding to the sample text data, wherein a first quality score of the first summary label is higher than a second quality score of the second summary label;

[0363] Inputting the sample text data into the updated preset neural network model, obtaining at least one second prediction probability sequence corresponding to at least one first candidate word included in the first summary label and at least one third prediction probability sequence corresponding to at least one second candidate word included in the second summary label output by the preset neural network model;

[0364] calculating a third loss based on the first summary label, the second summary label, the second predicted probability sequence, and the third predicted probability sequence;

[0365] Obtaining a second reference probability sequence corresponding to each first candidate word, and calculating a fourth loss based on the second predicted probability sequence and the second reference probability sequence corresponding to each first candidate word, and the third predicted probability sequence corresponding to each second candidate word;

[0366] A second target loss is calculated according to the third loss and the fourth loss, and the parameters of the preset neural network model are updated based on the second target loss to determine the target neural network model.

[0367] Optionally, in one implementation, calculating the fourth loss based on the second predicted probability sequence and the second reference probability sequence corresponding to each first candidate word, and the third predicted probability sequence corresponding to each second candidate word includes:

[0368] Calculate the first candidate word difference item corresponding to the first summary label based on the second prediction probability sequence corresponding to each first candidate word and the second reference probability sequence;

[0369] Calculate the second candidate word difference item corresponding to the second summary label based on the third prediction probability sequence corresponding to each second candidate word;

[0370] The fourth loss is calculated based on the difference item of the first candidate word and the difference item of the second candidate word.

[0371] Optionally, in one implementation, calculating the fourth loss according to the difference item of the first candidate word and the difference item of the second candidate word includes:

[0372] Determining a total difference term based on the first candidate word difference term and the second candidate word difference term;

[0373] The total difference term is normalized to obtain the fourth loss.

[0374] Optionally, in one implementation, determining a total difference item based on the first candidate word difference item and the second candidate word difference item includes:

[0375] Obtaining a first adjustment parameter corresponding to the first candidate word difference item and a second adjustment parameter corresponding to the second candidate word difference item;

[0376] A total difference term is calculated based on the first adjustment parameter, the second adjustment parameter, the first candidate word difference term and the second candidate word difference term.

[0377] Optionally, in one implementation, calculating the first target loss according to the first loss and the second loss includes:

[0378] Obtaining a first loss weight corresponding to the first loss and obtaining a second loss weight corresponding to the second loss;

[0379] The first loss and the second loss are weightedly calculated based on the first loss weight and the second loss weight to obtain the first target loss.

[0380] Optionally, in one implementation, obtaining a second loss weight corresponding to the second loss includes:

[0381] Get the weight curve corresponding to the second loss;

[0382] Based on the iterative rounds of updating the parameters of the preset neural network model during the training cycle, a second loss weight is determined from the weight curve.

[0383] Reference Fig.13 , Fig.13 To implement the structural block diagram of part of the terminal 140 of the embodiment of the present disclosure, the terminal includes: a radio frequency (RF) circuit 1310, a memory 1315, an input unit 1330, a display unit 1340, a sensor 1350, an audio circuit 1360, a wireless fidelity (WiFi) module 1370, a processor 1380, and a power supply 1390. Those skilled in the art can understand that Fig.13The structure of the terminal 140 shown does not constitute a limitation on a mobile phone or a computer, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0384] The RF circuit 1310 may be used for receiving and sending signals during information transmission or communication. In particular, after receiving downlink information from the base station, the information is sent to the processor 1380 for processing. In addition, the uplink data is sent to the base station.

[0385] The memory 1315 may be used to store software programs and modules. The processor 1380 executes the software programs and modules stored in the memory 1315 to execute various functional applications of the content terminal and generate text summaries.

[0386] The input unit 1330 may be used to receive input digital or character information and generate key signal input related to the setting and function control of the content terminal. Specifically, the input unit 1330 may include a touch panel 1331 and other input devices 1332 .

[0387] The display unit 1340 may be used to display input information or provided information and various menus of the content terminal. The display unit 1340 may include a display panel 1341.

[0388] The audio circuit 1360 , the speaker 1361 , and the microphone 1362 may provide an audio interface.

[0389] In this embodiment, the processor 1380 included in the target terminal 140 can execute the text summary generation method of the previous embodiment.

[0390] The embodiments of the present invention can be applied to various scenarios, including but not limited to intelligent text generation, text recommendation, etc.

[0391] Fig.14The structural block diagram of part of the server 110 for implementing the embodiment of the present disclosure. The server 110 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 1422 (for example, one or more processors) and memory 1432, and one or more storage media 1430 (for example, one or more mass storage devices) storing application programs 1442 or data 1444. Among them, the memory 1432 and the storage medium 1430 can be short-term storage or permanent storage. The program stored in the storage medium 1430 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 1422 can be configured to communicate with the storage medium 1430 and execute a series of instruction operations in the storage medium 1430 on the server.

[0392] The server 110 may also include one or more power supplies 1426, one or more wired or wireless network interfaces 1450, one or more input and output interfaces 1458, and / or, one or more operating systems 1441, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0393] The central processor 1422 in the server 110 may be used to execute the text summary generation method of the embodiment of the present disclosure.

[0394] The embodiments of the present disclosure also provide a computer-readable storage medium, which is used to store a computer program, and the computer program is used to execute the text summary generation method of each of the above embodiments.

[0395] The embodiment of the present disclosure also provides a computer program product, which includes a computer program. A processor of an electronic device reads and executes the computer program, so that the electronic device executes the above-mentioned text summary generation method.

[0396] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present disclosure and the above-mentioned drawings are used to distinguish similar contents, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "comprises" and "comprising" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0397] It should be understood that in the present disclosure, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated content, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the associated content before and after is in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0398] It should be understood that in the description of the embodiments of the present disclosure, the meaning of multiple (or multiple items) is more than two, greater than, less than, exceed, etc. are understood to not include the number, and above, below, within, etc. are understood to include the number.

[0399] In the embodiments of the present disclosure, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0400] In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0401] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0402] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0403] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0404] It should also be understood that the various implementations provided in the embodiments of the present disclosure can be combined arbitrarily to achieve different technical effects.

[0405] The above is a specific description of the implementation methods of the present disclosure, but the present disclosure is not limited to the above implementation methods. Technical personnel familiar with the art can also make various equivalent modifications or substitutions without violating the spirit of the present disclosure. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present disclosure.

Claims

1. A text summary generation method, characterized in that: include: Obtain target text and summary generation prompt text; Inputting the target text and the summary generation prompt text into a pre-trained target neural network model to obtain a text summary corresponding to the target text; The training process of the target neural network model includes: Acquire training sample data, where the training sample data includes sample text data and a first summary label corresponding to the sample text data; Inputting the sample text data into a preset neural network model, obtaining at least one first prediction probability sequence output by the preset neural network model and corresponding to at least one first candidate word included in the first summary label, wherein the first prediction probability sequence includes output probabilities corresponding to a plurality of reference words in a reference word library, and calculating a first loss based on the first summary label and the at least one first prediction probability sequence; Obtaining a first reference probability sequence corresponding to each of the first candidate words, and calculating a second loss based on the first predicted probability sequence corresponding to each of the first candidate words and the first reference probability sequence, wherein a first balance corresponding to a first probability distribution of the first reference probability sequence is higher than a second balance corresponding to a second probability distribution of the first predicted probability sequence; A first target loss is calculated according to the first loss and the second loss, and the parameters of the preset neural network model are updated based on the first target loss, and the target neural network model is determined based on the updated preset neural network model.

2. The method according to claim 1, characterized in that The calculating the second loss based on the first predicted probability sequence corresponding to each of the first candidate words and the first reference probability sequence includes: Calculate a first difference item corresponding to each of the first candidate words based on the first predicted probability sequence corresponding to each of the first candidate words and the first reference probability sequence; A second loss is calculated based on the first difference term.

3. The method according to claim 2, characterized in that The first summary tag includes at least two first candidate words. Before calculating the second loss based on the first difference item, the method further includes: Determine at least one candidate word sequence based on at least two first candidate words included in the first summary tag; Calculate a second difference item corresponding to each of the candidate word sequences based on the first predicted probability sequence corresponding to each of the first candidate words in each of the candidate word sequences and the corresponding first reference probability sequence; The calculating the second loss based on the first difference term includes: A second loss is calculated based on the first difference term and the second difference term.

4. The method according to claim 3, characterized in that The calculating the second difference item corresponding to each candidate word sequence based on the first predicted probability sequence corresponding to each first candidate word in each candidate word sequence and the corresponding first reference probability sequence includes: Generate a sequence prediction probability distribution based on the first prediction probability sequence corresponding to each of the first candidate words in each of the candidate word sequences, and generate a sequence reference probability distribution based on the first reference probability sequence corresponding to each of the first candidate words in the candidate word sequence; The second difference item corresponding to each candidate word sequence is calculated according to the sequence prediction probability distribution corresponding to each candidate word sequence and the sequence reference probability distribution.

5. The method according to claim 4, characterized in that The generating a sequence prediction probability distribution based on the first prediction probability sequence corresponding to each first candidate word in each candidate word sequence includes: splicing the first predicted probability sequences corresponding to each first candidate word in each candidate word sequence according to the arrangement order of the first candidate words to obtain a spliced ​​sequence; A sequence prediction probability distribution is generated based on the concatenated sequence and the number of first candidate words included in the candidate word sequence.

6. The method according to claim 1, characterized in that The step of determining the target neural network model based on the updated preset neural network model includes: Acquire a second summary tag corresponding to the sample text data, wherein a first quality score of the first summary tag is higher than a second quality score of the second summary tag; Inputting the sample text data into the updated preset neural network model, obtaining at least one second prediction probability sequence corresponding to the at least one first candidate word included in the first summary label and at least one third prediction probability sequence corresponding to the at least one second candidate word included in the second summary label output by the preset neural network model; calculating a third loss based on the first summary label, the second summary label, the second predicted probability sequence, and the third predicted probability sequence; Obtain a second reference probability sequence corresponding to each of the first candidate words, and calculate a fourth loss based on the second predicted probability sequence and the second reference probability sequence corresponding to each of the first candidate words, and the third predicted probability sequence corresponding to each of the second candidate words; A second target loss is calculated according to the third loss and the fourth loss, and the parameters of the preset neural network model are updated based on the second target loss to determine the target neural network model.

7. The method according to claim 6, characterized in that The calculating the fourth loss based on the second predicted probability sequence and the second reference probability sequence corresponding to each of the first candidate words, and the third predicted probability sequence corresponding to each of the second candidate words, comprises: Calculate a first candidate word difference item corresponding to the first summary label based on the second predicted probability sequence corresponding to each first candidate word and the second reference probability sequence; Calculate the second candidate word difference item corresponding to the second summary label based on the third prediction probability sequence corresponding to each of the second candidate words; A fourth loss is calculated based on the first candidate word difference item and the second candidate word difference item.

8. The method according to claim 7, characterized in that The calculating the fourth loss according to the first candidate word difference item and the second candidate word difference item includes: Determining a total difference item based on the first candidate word difference item and the second candidate word difference item; The total difference term is normalized to obtain the fourth loss.

9. The method according to claim 8, characterized in that The determining of the total difference item based on the first candidate word difference item and the second candidate word difference item comprises: Obtaining a first adjustment parameter corresponding to the first candidate word difference item and a second adjustment parameter corresponding to the second candidate word difference item; A total difference item is calculated based on the first adjustment parameter, the second adjustment parameter, the first candidate word difference item, and the second candidate word difference item.

10. The method according to claim 1, characterized in that The calculating a first target loss according to the first loss and the second loss includes: Obtaining a first loss weight corresponding to the first loss and obtaining a second loss weight corresponding to the second loss; The first loss and the second loss are weightedly calculated based on the first loss weight and the second loss weight to obtain a first target loss.

11. The method according to claim 10, characterized in that The obtaining a second loss weight corresponding to the second loss includes: Obtaining a weight curve corresponding to the second loss; Based on the iterative rounds of updating the parameters of the preset neural network model during the training cycle, a second loss weight is determined from the weight curve.

12. A text summary generation device, characterized in that: include: An acquisition unit, used for acquiring a target text and a summary generation prompt text; A generating unit, configured to input the target text and the summary generating prompt text into a pre-trained target neural network model to obtain a text summary corresponding to the target text; The training process of the target neural network model includes: Acquire training sample data, where the training sample data includes sample text data and a first summary label corresponding to the sample text data; Inputting the sample text data into a preset neural network model, obtaining at least one first prediction probability sequence output by the preset neural network model and corresponding to at least one first candidate word included in the first summary label, wherein the first prediction probability sequence includes output probabilities corresponding to a plurality of reference words in a reference word library, and calculating a first loss based on the first summary label and the at least one first prediction probability sequence; Obtaining a first reference probability sequence corresponding to each of the first candidate words, and calculating a second loss based on the first predicted probability sequence corresponding to each of the first candidate words and the first reference probability sequence, wherein a first balance corresponding to a first probability distribution of the first reference probability sequence is higher than a second balance corresponding to a second probability distribution of the first predicted probability sequence; A first target loss is calculated according to the first loss and the second loss, and the parameters of the preset neural network model are updated based on the first target loss, and the target neural network model is determined based on the updated preset neural network model.

13. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the text summary generating method according to any one of claims 1 to 11 is implemented.

14. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the text summary generation method according to any one of claims 1 to 11 is implemented.

15. A computer program product, comprising a computer program, wherein the computer program is read and executed by a processor of an electronic device, so that the electronic device executes the text summary generation method according to any one of claims 1 to 11.

Images