Text generation method and device, computer equipment and computer readable storage medium

Abstract

The invention discloses a text generation method and device, computer equipment and a computer readable storage medium. The method comprises the steps of obtaining a to-be-processed text and pre-constructed prompt information; performing plot and time relation extraction on the to-be-processed text based on the prompt information to obtain first time plot information of the to-be-processed text; and generating a target interpolation text of the to-be-processed text based on the first time plot information and the to-be-processed text. According to the method and the device, the illustrated plot text expression which better conforms to logic and has more drama and attraction can be generated.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and in particular to a text generation method, apparatus, computer device and computer-readable storage medium. Background Art

[0002] In plot generation games, players play games or interact with non-player characters (NPCs) through game plots, and the game plots are composed of plot texts. Therefore, plot texts need to be generated in plot generation games. Traditional plot generation games that pre-set plot texts manually have problems such as repeated plots, established patterns, fixed routines, and lack of novelty. In order to solve the above problems of plot generation games, people began to use artificial intelligence (AI) to generate plot texts. However, the plot texts generated by existing AI-based plot text generation methods have the problem that the previous and subsequent plots do not conform to the chronological order and logic. Summary of the invention

[0003] The embodiments of the present application provide a text generation method, apparatus, computer device and computer-readable storage medium, which can generate a more logical, dramatic and attractive flashback plot text expression.

[0004] The technical solution adopted by the present invention to solve the problem is as follows:

[0005] On the one hand, the present application provides a text generation method, comprising:

[0006] Get the text to be processed and pre-built prompt information;

[0007] Extract the plot and time relationship of the text to be processed based on the prompt information to obtain the first time plot information of the text to be processed;

[0008] Based on the first-time plot information and the text to be processed, a target inserted text of the text to be processed is generated.

[0009] In some implementation schemes of the present application, the plot and time relationship of the text to be processed are extracted based on the prompt information to obtain the first time plot information of the text to be processed, including:

[0010] The prompt information and the text to be processed are input into the plot generation model, and the plot generation model is used to extract the plot and time relationship of the text to be processed based on the prompt information to obtain the first-time plot information of the text to be processed.

[0011] In some embodiments of the present application, based on the first-time plot information and the text to be processed, generating a target interpolated text of the text to be processed includes:

[0012] Reconstructing the first time plot information and the text to be processed to obtain a first reconstructed text;

[0013] The first reconstructed text is input into the flashback generation model, and the flashback generation model outputs a target flashback text of the text to be processed.

[0014] In some implementation schemes of the present application, before obtaining the text to be processed and the pre-built prompt information, the following steps are included:

[0015] According to the target insert text generation task, a prompt template is constructed, wherein the prompt template includes a plurality of mask slots and a time relationship between the plurality of mask slots;

[0016] A plurality of mask slots in the prompt template are replaced with template slots of the structured plot to obtain prompt information, wherein the first-time plot information is the prompt information after the template slots are filled with the plot of the text to be processed.

[0017] In some embodiments of the present application, before generating a target inserted text of the text to be processed based on the first-time plot information and the text to be processed, the method includes:

[0018] Obtaining a training sample set, the training sample set including a training text and an inserted text of the training text;

[0019] Extract the plot and time relationship of the training text based on the prompt information to obtain the second time plot information of the training text;

[0020] The first basic model is trained based on the second time plot information, the training text and the inserted text to obtain the inserted text generation model.

[0021] In some implementation schemes of the present application, the plot and time relationship of the training text is extracted based on the prompt information to obtain the second time plot information of the training text, including:

[0022] Training the pre-trained model based on the training text to obtain a trained pre-trained model;

[0023] Determine a second basic model based on the trained pre-trained model;

[0024] The training text and the prompt information are input into the second basic model, and the second time plot information of the training text is output through the second basic model.

[0025] In some embodiments of the present application, the second basic model further outputs a third conditional probability of the plot label of the training text, and the first basic model is trained based on the second time plot information, the training text and the inserted text to obtain an inserted text generation model, including:

[0026] Reconstructing the second time plot information and the training text to obtain a second reconstructed text;

[0027] Inputting the second reconstructed text into the first basic model, and outputting the second conditional probability of the inserted text through the first basic model;

[0028] When the training situation of the first basic model does not meet the preset second condition, the model parameters of the first basic model and the model parameters of the second basic model are corrected based on the second conditional probability and the third conditional probability, and the step of inputting the training text and prompt information into the second basic model is continued until the training situation of the first basic model meets the second condition to obtain the flashback generation model.

[0029] In a second aspect, an embodiment of the present invention further provides a text generation device, comprising:

[0030] A text acquisition unit, used to acquire the text to be processed and pre-built prompt information;

[0031] An information extraction unit, used for extracting the plot and time relationship of the text to be processed based on the prompt information, and obtaining the first time plot information of the text to be processed;

[0032] The text processing unit is used to generate a target inserted text of the text to be processed based on the first-time plot information and the text to be processed.

[0033] In a third aspect, the present application further provides a computer device, the computer device comprising:

[0034] one or more processors;

[0035] Memory; and

[0036] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement any one of the text generation methods in the first aspect.

[0037] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and the computer program is loaded by a processor to execute the steps in any one of the text generation methods in the first aspect.

[0038] The beneficial effects of the present invention are as follows: based on the prompt information, the plot and time relationship of the text to be processed are extracted, and then based on the extracted first-time plot information, the target inserted text of the text to be processed is generated. The inserted text is generated by using the plot and time information of the text to be processed, so that a more logical, dramatic and attractive inserted plot text expression can be generated. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 It is a flowchart of a text generation method provided by an embodiment of the present invention;

[0041] Figure 2 is a flowchart of a specific embodiment of the text generation method provided by an embodiment of the present invention;

[0042] Figure 3 is a flowchart of a specific embodiment of a generation process of a flashback generation model provided by an embodiment of the present invention;

[0043] Figure 4 is a principle block diagram of a text generation device provided by an embodiment of the present invention;

[0044] Figure 5 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0046] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, which are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present application. In addition, the terms "first", "second", "third", "fourth", "fifth", "sixth", "seventh" and the like are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first", "second", "third", "fourth", "fifth", "sixth", "seventh" and the like may explicitly or implicitly include one or more of the features. In the description of the present application, "multiple" means two or more, and "several" means one or more, unless otherwise clearly and specifically defined.

[0047] In this application, the word "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described in this application as "exemplary" is not necessarily to be construed as being preferred or advantageous over other embodiments. The following description is given to enable any technician in the field to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present application can be implemented without using these specific details. In other instances, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.

[0048] It should be noted that since the method of the embodiment of the present application is executed in a computer device, the processing objects of each computer device exist in the form of data or information. For example, time is actually time information. It can be understood that if size, quantity, position, etc. are mentioned in subsequent embodiments, they are all corresponding data for processing by the computer device. The details will not be repeated here.

[0049] The inventor has found through research that traditional plot generation games are played by players directly selecting plots according to preset options. Therefore, plot generation games have problems such as repeated plots, established patterns, fixed routines, and lack of novelty. Artificial Intelligence (AI) creation can generate a series of new plots based on the story beginning or text outline input by the player, thereby getting rid of the limitations of the preset framework and increasing the novelty and challenge of the game. However, the existing AI-based plot text generation method has the following two problems: ① Since the plot game is developing towards a more complex story structure and a longer plot setting, the information extraction and text generation capabilities of the existing model are limited, and it is impossible to generate a plot that conforms to the content logic and time sequence according to the development of the plot; ② The generated text content and quality are not high, and there are many descriptions of background and scene atmosphere that are irrelevant to the plot, and most of the generated plots meet the most basic logical requirements, are straightforward, and lack dramatic expression. The above two points weaken the fluency, coherence and fun of the game, and reduce the player's gaming experience.

[0050] Based on this, in an embodiment of the present application, the text to be processed and pre-constructed prompt information are obtained; based on the prompt information, the plot and time relationship of the text to be processed are extracted to obtain the first-time plot information of the text to be processed; based on the first-time plot information and the text to be processed, a target flashback text of the text to be processed is generated, which can generate a more logical, dramatic and attractive flashback plot text expression.

[0051] The content of this application is further illustrated below through the description of embodiments in conjunction with the accompanying drawings.

[0052] This embodiment provides a text generation method, such as Figure 1 As shown in , the method includes:

[0053] Step S10: Obtain the text to be processed and pre-built prompt information.

[0054] The text to be processed is the text for which the flashback needs to be generated. The text to be processed can be the first sentence, title or introductory sentence of the story of the plot game. The text to be recognized can be a sentence or a paragraph input by the user through a computer device (for example, a smart phone), or can be text data obtained by converting the collected user dialogue into text through a computer device. For example, the text to be processed is "Jack had a car accident and he lost his consciousness. He woke up in the morning."

[0055] The prompt information is a pre-built prompt template with time and plot. The prompt information can be expressed as x in =[ea1 ]…[e ai ] <after>[e b1 ]…[e bj ], where [e a1 ]…[e ai ] and [e b1 ]…[e bj ] indicates the plot, <after>Indicates the time relationship, this formula represents the plot [e a1 ]…[e ai ]In the plot [e b1 ]…[e bj ]After that. Episode e ai It can be expressed as a trigger t ai And two parameters and Episode bj It can be expressed as a trigger t bj And two parameters and The prompt information can be further expressed as

[0056] In a specific implementation, before step S10, the following steps are included:

[0057] Step S01: construct a prompt template according to the target insert text generation task, wherein the prompt template includes a plurality of mask slots and the time relationship between the plurality of mask slots;

[0058] Step S02: Replace multiple mask slots in the prompt template with template slots of the structured plot to obtain prompt information.

[0059] In a specific implementation, Figure 2 As shown in the figure, when constructing the prompt information, we can first construct a prompt template according to the characteristics of the target insert text generation task, such as the prompt template is x prompt =[MASK] a <after>[MASK] b , the prompt template represents [MASK] a In [MASK] b After that, the prompt template conforms to the logic of the usual flashback writing. After the prompt template is constructed, the [MASK] slot in the template form can be replaced with the template slot [e k ](Right now ), which constitutes a prompt message with time and plot hints x in =[e a1 ]…[e ai ] <before>[e b1 ]…[e bj ].

[0060] Step S20: extracting the plot and time relationship of the text to be processed based on the prompt information to obtain the first time plot information of the text to be processed.

[0061] Plot refers to the words in a sentence that can reflect the performance and process of an event. Plot can be represented by a trigger and two parameters. The i-th sentence S in the training text i The plot i It can be expressed as: Among them, t i Represents a trigger, and Represents two parameters, <eoe>Indicates the end of an event. For example, the sentence He drove a car on the road can be represented as "drove; He; car", where drove represents the trigger, and He and car represent two parameters.

[0062] The temporal relationship is the temporal relationship between multiple plots. The temporal relationship includes three relationships: the i-th plot is before the i+1th plot, the i-th plot is after the i+1th plot, and the temporal order between the i-th plot and the i+1th plot is uncertain. When the i-th plot is before the i+1th plot, the temporal relationship can be expressed as r i = <before>; When the i-th episode comes after the i+1-th episode, the temporal relationship can be expressed as r i = <after>, when the time order between the i-th episode and the i+1-th episode is uncertain, the time relationship can be expressed as r i = <vague>.

[0063] The first time plot information is the prompt information after the template slots of the prompt information are filled with the plot of the text to be processed. The first time plot information includes the plot of the text to be processed and the time relationship between the plots. Figure 2 As shown in FIG. 1 , based on the prompt information, the plot and time relationship of the processed text "Jack had a car accident and he lost his consciousness. He woke up in the morning." are extracted, and the first time plot information obtained is [woke up; he] <after>[had; he; car accident] [lost; he; consciousness].

[0064] In a specific implementation, step S20 includes:

[0065] S21, inputting the prompt information and the text to be processed into a plot generation model, extracting the plot and time relationship of the text to be processed based on the prompt information through the plot generation model, and obtaining the first time plot information of the text to be processed.

[0066] The plot generation model is a pre-trained network model for extracting plots and time relationships from the text to be processed based on prompt information. After obtaining the text to be processed and the pre-built prompt information, the prompt information and the text to be processed can be input into the plot generation model, and the template slots [e k ] and fill in the template slots based on the predicted plot, thereby obtaining the first-time plot information of the text to be processed. Figure 2 As shown in FIG. 1 , the text to be processed "Jack had a car accident and he lost his consciousness. He woke up in the morning." and the prompt information are input into the plot generation model, and the plot generation model outputs the plot prompt information x out =[woke up; he] <after>[had; he; car accident] [lost; he; consciousness].

[0067] Step S30: Generate a target inserted text of the text to be processed based on the first time plot information and the text to be processed.

[0068] The target inserted text is an inserted plot description of the text to be processed generated based on the first-time plot information. This embodiment generates the inserted text based on the plot prompt information of the text to be processed, and can generate an inserted plot text expression that is more logical, dramatic and attractive.

[0069] In a specific implementation, step S30 includes:

[0070] Step S31, reconstructing the first time plot information and the text to be processed to obtain a first reconstructed text;

[0071] Step S32: input the first reconstructed text into the flashback generation model, and output the target flashback text of the text to be processed through the flashback generation model.

[0072] The first reconstructed text is a text obtained by reconstructing the first time plot information and the text to be processed. The first reconstructed text can be expressed as X1=[CLS]x out1 [X][SEP], where X1 represents the first reconstructed text, X represents the text to be processed, and x out1 Indicates the first time plot information, [CLS] indicates the start identifier, and [SEP] indicates the separator identifier.

[0073] The interpolation generation model is a pre-trained network model for generating a target interpolation text of the text to be processed based on the first reconstructed text. After obtaining the first reconstructed text, this embodiment inputs the first reconstructed text into the interpolation generation model, and outputs the target interpolation text of the text to be processed through the interpolation generation model. Figure 2 As shown, the plot generation model outputs temporal plot information x out After that, the text to be processed and the time plot information\reconstruction are used as the input of the flashback generation model, and the flashback generation model outputs the target text "John woke up in the hospital in the morning. He struggled to remember that he had a car accident, and he lost consciousness when the ambulance arrived.".

[0074] In a specific implementation, step S30 includes:

[0075] Step S40, obtaining a training sample set, wherein the training sample set includes a training text and an inserted text of the training text;

[0076] Step S50, extracting the plot and time relationship of the training text based on the prompt information to obtain the second time plot information of the training text;

[0077] Step S60: training the first basic model based on the second time plot information, the training text and the inserted text to obtain an inserted text generation model.

[0078] The training sample set is a sample set preset for training the first basic model. The training sample set includes the training text and the interpolated text of the training text. The interpolated text is the text obtained by inserting one or more plots or events related to the central event into the training text without changing the central point of the full text of the training text. For example, the training text is "Jack had a car accident and he lost his consciousness. He woke up in the morning.", and the interpolated text corresponding to the training text is "John woke up in the hospital in the morning. He struggled to remember that he had a car accident, and he lost consciousness when the ambulance arrived."

[0079] The second time plot information is the prompt information after the template slots of the prompt information are filled with the plot of the training text. The second time plot information includes the plot of the training text and the time relationship between the plots. The flashback generation model is a model obtained by training the first basic model based on the second time plot information, the training text and the flashback text. The structure of the flashback generation model is the same as that of the first basic model. The difference between the flashback generation model and the first basic model is that the model parameters of the first basic model are initial parameters, while the model parameters of the flashback generation model are the model parameters of the trained first basic model. Since the flashback generation model is trained based on the second time plot information, the obtained flashback generation model can generate a more logical, dramatic and attractive flashback plot text expression.

[0080] In a specific implementation, Figure 3 As shown, step S50 includes:

[0081] Step S51: training the pre-training model based on the training text to obtain a trained pre-training model;

[0082] Step S52: Determine a second basic model based on the trained pre-trained model;

[0083] Step S53: input the training text and the prompt information into the second basic model, and output the second time plot information of the training text through the second basic model.

[0084] The pre-trained model is a trained model. The purpose of training the pre-trained model based on the training text is to fine-tune the pre-trained model with the training text so that the pre-trained model can better match downstream tasks (such as the flashback generation task). For example, the pre-trained model can use a generative pre-trained Transformer (GPT) model, such as the gpt2-large model. The second basic model is used to extract the plot and time relationship of the text. The structure of the second basic model is the same as that of the pre-trained model, and the model parameters of the second basic model are the model parameters of the trained pre-trained model.

[0085] In a specific implementation, the second basic model is used to extract the plot and time relationship of the training text. Accordingly, the step of extracting the plot and time relationship of the training text specifically includes: inputting the training text and the prompt information into the second basic model, predicting the template slot [e k ] and fill in the template slots based on the predicted plot, thereby obtaining the second time plot information of the training text. In other words, the second time plot information is the prompt information after the template slots are filled in.

[0086] In a specific implementation, the second basic model adopts a pre-trained model (such as a GPT model). In order to enhance the expressive power of the pre-trained model, before inputting the training text and prompt information into the second basic model, the present embodiment first trains the pre-trained model based on the training text to obtain a trained pre-trained model, and then determines the second basic model based on the trained pre-trained model. When determining the second basic model based on the trained pre-trained model, the trained pre-trained model can be directly used as the second basic model, or a model with the same structure as the pre-trained model can be selected, and the model parameters of the trained pre-trained model can be used as the model parameters of the model to obtain the second basic model.

[0087] Furthermore, in this embodiment, the structure of the first basic model is the same as the structure of the pre-trained model, and the model parameters of the first basic model are the model parameters of the trained pre-trained model. By using the model parameters of the trained pre-trained model as the model parameters of the first basic model and the second basic model, a good initialization can be provided for the training of the first basic model and the second basic model.

[0088] In a specific implementation, continue to refer to Figure 3 As shown, step S51 includes:

[0089] Step S511, performing semantic analysis on the training text to obtain plot information of the training text;

[0090] Step S512, extracting the time relationship of the plot information to obtain the time relationship information of the plot information;

[0091] Step S513: train the pre-trained model based on the training text, plot information and time relationship information to obtain a trained pre-trained model.

[0092] The plot information is the information obtained by semantically analyzing the training text based on the semantic analysis tool. The plot information contains the plot of each sentence in the training text. The plot information can be expressed as E = e1, e2, ..., e n , where e i Represents the i-th sentence S in the training text i For example, semantic role labeling (SRL) is used to perform semantic analysis on the text "Jack had a car accident and he lost his consciousness. He woke up in the morning." The plot information obtained includes [woke up; he], [had; he; car accident] and [lost; he; consciousness].

[0093] The temporal relationship information is the information obtained by extracting the temporal relationship of the plot information of the training text based on the temporal relationship extraction tool. The temporal relationship information includes the temporal relationship between the plots in the plot information. For example, by using the temporal relationship extraction tool to extract the temporal relationship of the plot information of the text "Jack had a car accident and he lost his consciousness. He woke up in the morning.", the temporal relationship information [wokeup; he] can be obtained. <after>[had; he; car accident][lost; he; consciousness], this time relationship information indicates that the plot [woke up; he] comes after the plot [had; he; car accident] and the plot [lost; he; consciousness].

[0094] When the present embodiment trains the pre-trained model based on the training text, the training text is firstly semantically analyzed by a semantic analysis tool (such as SRL) to obtain the plot information of the training text, and then the temporal relationship extraction tool is used to extract the temporal relationship of the plot information to obtain the temporal relationship information of the plot information, and then the pre-trained model is trained based on the training text, the plot information and the temporal relationship information to obtain the trained pre-trained model. By structuring the story plot by the semantic analysis tool and the temporal relationship extraction tool, and then using the structured story plot as enhanced data to fine-tune the pre-trained model, the pre-trained model tasks can be matched to the downstream tasks, and the temporal relationship between adjacent events can be captured, which is conducive to the model to understand the overall development trend of the story while understanding the context of the story, so that the model can better generate content that conforms to the content logic according to the direction of the story.

[0095] In a specific implementation, step S513 includes:

[0096] Step S5131, splicing the training text, plot information and time relationship information to obtain a spliced ​​text;

[0097] Step S5132: input the concatenated text into the pre-trained model, and output the first conditional probability of each word in the concatenated text through the pre-trained model;

[0098] Step S5133: modify the model parameters of the pre-trained model based on the first conditional probability, and continue to execute the step of inputting the concatenated text into the pre-trained model until the training status of the pre-trained model meets the preset first condition.

[0099] The concatenated text is the text obtained by concatenating the training text, plot information and time relationship information, where the i-th sentence P in the concatenated text is i It can be expressed as: i =concat(S i ,e i ,r i ), S i represents the i-th sentence in the training text, e i represents the plot of the i-th sentence, r i Indicates the plot i With plot e i+1 The time relationship of the i-th sentence P in the training text i The first conditional probability of the t-th word in the equation represents the probability of predicting the t-th word given the previous t-1 words. The first conditional probability can be expressed as P(x i,t |x i,t-1 ,…,x i,1 ), where x i,t Represents the i-th sentence P i The tth word in .

[0100] In this embodiment, the concatenated text P=P1, P2, ..., P N After inputting the pre-trained model, the pre-trained model will perform sentence P of the concatenated text. i Decode and then decode the P i Perform linear operation, and then convert the P after linear operation i Acting on the preset activation function, we get P i The first conditional probability of each word in P. i The decoding process can be expressed as: i = TransformerDecoder(P i ).

[0101] The first condition is used to measure the training status of the pre-trained model. The preset first condition can be that the number of training times of the pre-trained model reaches the preset first number of iterations or the third loss value determined by the first conditional probability satisfies the preset third condition. When the preset first condition is that the third loss value determined by the first conditional probability satisfies the preset third condition, the third loss value can be first determined based on the first conditional probability, and then it can be determined whether the third loss value satisfies the preset third condition. When the third loss value does not meet the preset third condition, the model parameters of the pre-trained model are corrected based on the preset parameter learning rate, and the step of inputting the concatenated text into the pre-trained model is continued until the third loss value meets the preset third condition. Among them, the calculation formula of the third loss value is: represents the third loss value, N represents the number of sentences in the concatenated text, T i represents the length of each sentence in the concatenated text, P(x i,t |x i,t-1 ,…,x i,1 ) represents the i-th sentence P in the concatenated text i The first conditional probability of the tth word in .

[0102] In a specific implementation, the second basic model also outputs a third conditional probability of the plot label of the training text, and continues to refer to Figure 3 As shown, step S60 includes:

[0103] Step S61, reconstructing the second time plot information and the training text to obtain a second reconstructed text;

[0104] Step S62, inputting the second reconstructed text into the first basic model, and outputting the second conditional probability of the inserted text through the first basic model;

[0105] Step S63: When the training condition of the first basic model does not meet the preset second condition, the model parameters of the first basic model and the model parameters of the second basic model are corrected based on the second conditional probability and the third conditional probability, and the step of inputting the training text and prompt information into the second basic model is continued until the training condition of the first basic model meets the second condition to obtain the flashback generation model.

[0106] The third conditional probability is the probability of predicting the plot label based on the training text and the prompt information. The plot label of the training text is the real plot of the training text. After the training text and the prompt information are input into the second basic model in step S53, the second basic model will output the probability of each slot in the prompt information under multiple predicted plots. Based on the probability of each slot under multiple predicted plots, the third conditional probability of the plot label of the training text can be determined. For example, the second basic model predicts that slot A belongs to plot e a1 The probability is 0.2, belonging to episode e a2 The probability is 0.7, belonging to episode e a3 The probability is 0.1, and the plot label of the training text corresponding to slot A is e a2 , then the third conditional probability of the plot label of the training text corresponding to slot A is 0.7.

[0107] In order to prevent the second basic model from generating non-plot prediction words during the prediction process, this embodiment designs a constraint mechanism for the model to generate answers. Specifically, this embodiment extracts the plot of the preset story data set through a semantic analysis tool (such as SRL), and uses the plot as the knowledge base E of the prompt information. all =e1,e2,…,e t When the second basic model outputs the probability of each slot in the prompt information under multiple prediction scenarios, the prediction scenario e is first determined. i The trigger word t i Is the prompt information episode library E all In the trigger word set T in , if , the dot product of the decoding layer vector h and the predicted word embedding vector is applied to the Softmax function to obtain the probability of the slot under the predicted scenario; otherwise, when the predicted scenario e is determined i The trigger word t i Not in prompt information episode library E all In the trigger word set T in , the probability of the output slot under the predicted plot is 0, thereby constraining the predicted plot of the second basic model to the prompt information plot library E all Among them, the calculation formula for the probability of a slot in the predicted scenario is: P(t i =w|S) represents the generation prediction of the trigger word in the i-th slot of the template, h i represents the decoding layer vector, emb(w) represents the predicted word embedding vector, and t i Indicates trigger words, T indicates prompt information plot library E all The trigger words set.

[0108] The second reconstructed text is the text obtained by reconstructing the second time plot information and the training text. The second reconstructed text can be expressed as X2=[CLS]x out2 [S][SEP], where X2 represents the second reconstructed text, S represents the training text, and x out2 Indicates the second time plot information, [CLS] indicates the start identifier, and [SEP] indicates the separation identifier.

[0109] The second conditional probability represents the probability of predicting the inserted text based on the second reconstructed text. The second condition can be that the number of training times of the first basic model meets the preset second number of iterations or the first loss value determined based on the second conditional probability meets the preset fourth condition. When the second condition is that the first loss value determined based on the second conditional probability meets the preset fourth condition, after the second conditional probability of the inserted text is output through the first basic model, the first loss value is determined based on the second conditional probability, and then it is determined whether the first loss value meets the fourth condition. When the first loss value does not meet the fourth condition, the model parameters of the first basic model and the model parameters of the second basic model are corrected based on the second conditional probability and the third conditional probability, and the step of inputting the training text and the prompt information into the second basic model is continued until the first loss value meets the fourth condition. This embodiment conducts joint training of the first basic model and the second basic model, which can alleviate the gap between training and reasoning, so that the inserted text generation model can generate more continuous and reasonable inserted text.

[0110] In a specific implementation, the step of correcting the model parameters of the first basic model and the model parameters of the second basic model based on the second conditional probability and the third conditional probability in step S63 includes:

[0111] Step S631, determining a first loss value based on the second conditional probability, and determining a second loss value based on the third conditional probability;

[0112] Step S632: determining an expected reward of a second basic model based on the first loss value and the second loss value;

[0113] Step S633: determining a first update gradient of the first basic model based on the first loss value, and determining a second update gradient of the second basic model based on the expected reward;

[0114] Step S634: update the model parameters of the first basic model based on the first update gradient, and update the model parameters of the second basic model based on the second update gradient.

[0115] The first loss value is the loss value of the first basic model determined based on the second conditional probability, and the second loss value is the loss value of the second basic model determined based on the third conditional probability. The calculation formula of the first loss value is: The calculation formula for the second loss value is: represents the first loss value, represents the second loss value, S represents the training text, X2 represents the second reconstructed text, E represents the set of episode labels of the training text, and e k represents the kth episode label in the episode label set, θ represents the model parameters of the first base model, and x in Indicates prompt information, y i represents the i-th sentence in the inserted text, and n represents the number of sentences in the inserted text.

[0116] Due to the loss of the first base model during training It cannot be directly back-propagated to the second basic model, resulting in the second basic model being unable to adjust according to the loss of the first basic model. In order to solve this problem, when this embodiment corrects the model parameters of the first basic model and the model parameters of the second basic model, it first determines the expected reward of the second basic model based on the first loss value and the second loss value, then determines the first update gradient of the first basic model based on the first loss value, and determines the second update gradient of the second basic model based on the expected reward, and finally updates the model parameters of the first basic model based on the first update gradient, and updates the model parameters of the second basic model based on the second update gradient. Among them, the update formula of the model parameters of the first basic model is: The updating formula of the model parameters of the second basic model is: θ represents the model parameters of the first basic model, represents the first update gradient, α represents the model parameters of the second basic model, represents the second updated gradient.

[0117] Further, when determining the expected reward of the second basic model based on the first loss value and the second loss value, the reward is first determined based on the first loss value, and then the expected reward of the second basic model is determined based on the reward and the second loss value. The calculation formula of the expected reward is: α [R] = O[R·log(P(E|S,x in ))], the calculation formula for the second updated gradient is: O α [R] represents the expected reward, R represents the reward, log P(E|S,x in ) is determined by the second loss value.

[0118] In order to better implement the text generation method in the embodiment of the present application, based on the text generation method, the embodiment of the present application also provides a text generation device, such as Figure 4 As shown, the text generation device includes:

[0119] The text acquisition unit 610 is used to acquire the text to be processed and the pre-built prompt information;

[0120] An information extraction unit 620 is used to extract the plot and time relationship of the text to be processed based on the prompt information to obtain the first time plot information of the text to be processed;

[0121] The text generation unit 630 is used to generate a target inserted text of the text to be processed based on the first time plot information and the text to be processed.

[0122] In the embodiment of the present application, the plot and time relationship of the text to be processed are extracted based on the prompt information, and then the target inserted text of the text to be processed is generated based on the extracted first-time plot information. The inserted text is generated by using the plot and time information of the text to be processed, so that a more logical, dramatic and attractive inserted plot text expression can be generated.

[0123] In some embodiments of the present application, the information extraction unit 620 is specifically used to:

[0124] The prompt information and the text to be processed are input into the plot generation model, and the plot generation model is used to extract the plot and time relationship of the text to be processed based on the prompt information to obtain the first-time plot information of the text to be processed.

[0125] In some embodiments of the present application, the text generation unit 630 is specifically used to:

[0126] Reconstructing the first time plot information and the text to be processed to obtain a first reconstructed text;

[0127] The first reconstructed text is input into the flashback generation model, and the flashback generation model outputs a target flashback text of the text to be processed.

[0128] In some embodiments of the present application, the text generation device further includes:

[0129] A sample acquisition unit, used to acquire a training sample set, wherein the training sample set includes a training text and an inserted text of the training text;

[0130] A plot extraction unit, used to extract the plot and time relationship of the training text based on the prompt information to obtain the second time plot information of the training text;

[0131] The model generation unit is used to train the first basic model based on the second time plot information, the training text and the inserted text to obtain the inserted text generation model.

[0132] In some embodiments of the present application, the plot extraction unit is specifically used to:

[0133] Training the pre-trained model based on the training text to obtain a trained pre-trained model;

[0134] Determine a second basic model based on the trained pre-trained model;

[0135] The training text and the prompt information are input into the second basic model, and the second time plot information of the training text is output through the second basic model.

[0136] In some embodiments of the present application, the plot extraction unit is further configured to:

[0137] Perform semantic analysis on the training text to obtain the plot information of the training text;

[0138] Extracting the temporal relationship of the plot information to obtain the temporal relationship information of the plot information;

[0139] The pre-trained model is trained based on the training text, the plot information and the time relationship information to obtain a trained pre-trained model.

[0140] In some embodiments of the present application, the plot extraction unit is further configured to:

[0141] splicing the training text, plot information and time relationship information to obtain a spliced ​​text;

[0142] Input the concatenated text into the pre-trained model, and output the first conditional probability of each word in the concatenated text through the pre-trained model;

[0143] The model parameters of the pre-trained model are modified based on the first conditional probability, and the step of inputting the concatenated text into the pre-trained model is continued until the training status of the pre-trained model meets the preset first condition.

[0144] In some embodiments of the present application, the second basic model further outputs a third conditional probability of the plot label of the training text, and the model generation unit is specifically used to:

[0145] Reconstructing the second time plot information and the training text to obtain a second reconstructed text;

[0146] Inputting the second reconstructed text into the first basic model, and outputting the second conditional probability of the inserted text through the first basic model;

[0147] When the training situation of the first basic model does not meet the preset second condition, the model parameters of the first basic model and the model parameters of the second basic model are corrected based on the second conditional probability and the third conditional probability, and the step of inputting the training text and prompt information into the second basic model is continued until the training situation of the first basic model meets the second condition.

[0148] In some embodiments of the present application, the model generation unit is further used to:

[0149] determining a first loss value based on the second conditional probability, and determining a second loss value based on the third conditional probability;

[0150] Determining an expected reward of a second base model based on the first loss value and the second loss value;

[0151] Determine a first update gradient for the first base model based on the first loss value, and determine a second update gradient for the second base model based on the expected reward;

[0152] The model parameters of the first base model are updated based on the first update gradient, and the model parameters of the second base model are updated based on the second update gradient.

[0153] The embodiment of the present application further provides a computer device, which integrates any text generation device provided in the embodiment of the present application, and the computer device includes:

[0154] one or more processors;

[0155] Memory; and

[0156] One or more applications, wherein the one or more applications are stored in a memory and are configured to execute, by a processor, the steps of the text generation method in any of the above-mentioned text generation method embodiments.

[0157] The present application also provides a computer device that integrates any of the text generation devices provided in the present application. Figure 5 As shown, it shows a schematic diagram of the structure of the computer device involved in the embodiment of the present application, specifically:

[0158] The computer device may include one or more processing core processors 801, one or more computer-readable storage media memories 802, a power supply 803, an input unit 804 and other components. Those skilled in the art will appreciate that Figure 5 The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0159] The processor 801 is the control center of the computer device. It uses various interfaces and lines to connect various parts of the entire computer device. By running or executing software programs and / or modules stored in the memory 802 and calling data stored in the memory 802, it executes various functions of the computer device and processes data, thereby monitoring the computer device as a whole. Optionally, the processor 801 may include one or more processing cores; preferably, the processor 801 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 801.

[0160] The memory 802 can be used to store software programs and modules. The processor 801 executes various functional applications and data processing by running the software programs and modules stored in the memory 802. The memory 802 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 802 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 802 may also include a memory controller to provide the processor 801 with access to the memory 802.

[0161] The computer device also includes a power supply 803 for supplying power to each component. Preferably, the power supply 803 can be logically connected to the processor 801 through a power management system, so that the power management system can manage charging, discharging, power consumption and other functions. The power supply 803 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0162] The computer device may further include an input unit 804, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0163] Although not shown, the computer device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 801 in the computer device will load the executable files corresponding to the processes of one or more application programs into the memory 802 according to the following instructions, and the processor 801 will run the application programs stored in the memory 802, thereby realizing various functions, as follows:

[0164] Get the text to be processed and pre-built prompt information;

[0165] Extract the plot and time relationship of the text to be processed based on the prompt information to obtain the first time plot information of the text to be processed;

[0166] Based on the first-time plot information and the text to be processed, a target inserted text of the text to be processed is generated.

[0167] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0168] To this end, the embodiment of the present application provides a computer-readable storage medium, which may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any text generation method provided in the embodiment of the present application. For example, the computer program is loaded by a processor to execute the following steps:

[0169] Get the text to be processed and pre-built prompt information;

[0170] Extract the plot and time relationship of the text to be processed based on the prompt information to obtain the first time plot information of the text to be processed;

[0171] Based on the first-time plot information and the text to be processed, a target inserted text of the text to be processed is generated.

[0172] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the detailed description of other embodiments above, and will not be repeated here.

[0173] In specific implementation, the above units or structures can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units or structures can refer to the previous method embodiments, which will not be repeated here.

[0174] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0175] The above is a detailed introduction to a text generation method, device, computer equipment and computer-readable storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.< / after> < / after> < / after> < / vague> < / after> < / before> < / eoe> < / before> < / after> < / after> < / after>

Claims

1. A text generation method, characterized in that: include: Get the text to be processed and pre-built prompt information; Extracting the plot and time relationship of the text to be processed based on the prompt information to obtain the first time plot information of the text to be processed; Based on the first time plot information and the text to be processed, a target inserted text of the text to be processed is generated.

2. The method according to claim 1, characterized in that: The extracting the plot and time relationship of the text to be processed based on the prompt information to obtain the first time plot information of the text to be processed includes: The prompt information and the text to be processed are input into a plot generation model, and the plot generation model extracts the plot and time relationship of the text to be processed based on the prompt information to obtain the first time plot information of the text to be processed.

3. The method according to claim 1, characterized in that The step of generating a target inserted text of the text to be processed based on the first time plot information and the text to be processed includes: Reconstructing the first time plot information and the text to be processed to obtain a first reconstructed text; The first reconstructed text is input into a flashback generation model, and the flashback generation model is used to output a target flashback text of the text to be processed.

4. The method according to claim 1, characterized in that Before obtaining the text to be processed and the pre-built prompt information, the following steps are included: According to the target inserted text generation task, a prompt template is constructed, wherein the prompt template includes a plurality of mask slots and a time relationship between the plurality of mask slots; The plurality of mask slots in the prompt template are replaced with template slots of the structured plot to obtain the prompt information, wherein the first time plot information is the prompt information after the template slots are filled with the plot of the text to be processed.

5. The method according to claim 3, characterized in that: Before generating a target inserted text of the text to be processed based on the first time plot information and the text to be processed, the method includes: Acquire a training sample set, wherein the training sample set includes a training text and an inserted text of the training text; Extracting the plot and time relationship of the training text based on the prompt information to obtain second time plot information of the training text; The first basic model is trained based on the second time plot information, the training text and the inserted text to obtain an inserted text generation model.

6. The method according to claim 5, characterized in that The extracting the plot and time relationship of the training text based on the prompt information to obtain the second time plot information of the training text includes: Training the pre-trained model based on the training text to obtain a trained pre-trained model; Determine a second basic model based on the trained pre-trained model; The training text and the prompt information are input into the second basic model, and the second time plot information of the training text is output through the second basic model.

7. The method according to claim 5, characterized in that The second basic model further outputs a third conditional probability of the plot label of the training text. The first basic model is trained based on the second temporal plot information, the training text and the inserted text to obtain an inserted text generation model, including: Reconstructing the second time plot information and the training text to obtain a second reconstructed text; Inputting the second reconstructed text into the first basic model, and outputting a second conditional probability of the inserted text through the first basic model; When the training situation of the first basic model does not meet the preset second condition, the model parameters of the first basic model and the model parameters of the second basic model are corrected based on the second conditional probability and the third conditional probability, and the step of inputting the training text and the prompt information into the second basic model is continued until the training situation of the first basic model meets the second condition to obtain the flashback generation model.

8. A text generation device, characterized in that: include: A text acquisition unit, used to acquire the text to be processed and pre-built prompt information; An information extraction unit, configured to extract the plot and time relationship of the text to be processed based on the prompt information, and obtain the first time plot information of the text to be processed; A text generation unit is used to generate a target inserted text of the text to be processed based on the first time plot information and the text to be processed.

9. A computer device, characterized in that: The computer device comprises: one or more processors; Memory; and One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the text generation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in the text generation method according to any one of claims 1 to 7.

Images