Text generation method and device, computer equipment and computer readable storage medium

Abstract

The invention discloses a text generation method and device, computer equipment and a computer readable storage medium. The method comprises the steps of obtaining a to-be-processed text and target prefix information of the to-be-processed text; fusing the to-be-processed text and the target prefix information to obtain target fusion information; and generating a target text of the to-be-processed text based on the target fusion information. By adopting the method provided by the invention, the target text conforming to the specific emotional attribute tag can be generated, so that the generated target text is more fit with the development of a story.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and in particular to a text generation method, apparatus, computer device and computer-readable storage medium. Background Art

[0002] In plot generation games, players play games or interact with non-player characters (NPCs) through game plots, and the game plots are composed of plot texts. Therefore, plot texts need to be generated in plot generation games. Existing plot text generation methods are usually based on pre-trained language models and pre-learned knowledge to generate fluent text content that meets the most basic logical requirements. The generated texts often do not have emotional attributes and are not consistent with the development of the game plot, which weakens the fluency, coherence and fun of the game and reduces the player's immersive experience. Summary of the invention

[0003] The embodiments of the present application provide a text generation method, apparatus, computer device, and computer-readable storage medium, which can generate a target text that conforms to a specific emotional attribute label, so that the generated target text is more consistent with the development of the plot.

[0004] The technical solution adopted by the present invention to solve the problem is as follows:

[0005] On the one hand, the present application provides a text generation method, comprising:

[0006] Obtain the text to be processed and the target prefix information of the text to be processed;

[0007] Fusing the text to be processed and the target prefix information to obtain target fusion information;

[0008] Based on the target fusion information, a target text of the text to be processed is generated.

[0009] In some embodiments of the present application, obtaining target prefix information of the text to be processed includes:

[0010] Obtain text material information and pre-trained models;

[0011] Based on the text material information and the pre-trained model, the sentiment-related information is determined; the sentiment-related information represents the correspondence between the sentiment information and the prefix information;

[0012] The target sentiment information of the target text is obtained, and the target prefix information of the text to be processed is determined based on the target sentiment information and the sentiment association information.

[0013] In some implementation schemes of the present application, the text material information includes training text and text emotion information of the training text. Based on the text material information and the pre-training model, the emotion-related information is determined, including:

[0014] Based on the text sentiment information, construct the initial prefix information;

[0015] Based on the training text and the pre-training model, the parameters of the initial prefix information are updated to obtain candidate prefix information corresponding to the text sentiment information;

[0016] Fine-tune the parameters of the candidate prefix information based on the training text and the pre-training model to obtain the fine-tuned prefix information corresponding to the text sentiment information;

[0017] Based on the text sentiment information and fine-tuned prefix information, the sentiment association information is determined.

[0018] In some implementation schemes of the present application, the parameters of the initial prefix information are updated based on the training text and the pre-training model to obtain candidate prefix information corresponding to the text sentiment information, including:

[0019] Fusing the training text and the initial prefix information to obtain first fused information;

[0020] Inputting the first fusion information into a pre-training model, and outputting first word distribution information of the training text through the pre-training model;

[0021] Based on the first word distribution information, construct a first candidate word set for the training text;

[0022] Determine a first word-level loss and a second word-level loss of the training text based on the first candidate word set;

[0023] The parameters of the initial prefix information are updated based on the first word-level loss and the second word-level loss to obtain candidate prefix information corresponding to the text sentiment information.

[0024] In some embodiments of the present application, the first candidate word set includes a plurality of first candidate words, and determining a first word-level loss and a second word-level loss of a training text based on the first candidate word set includes:

[0025] Construct reverse prompt information for each first candidate word;

[0026] Based on the reverse prompt information, determining a plurality of second candidate words corresponding to the plurality of first candidate words respectively, each second candidate word being an antonym of the first candidate word corresponding to each second candidate word;

[0027] Fusing each first candidate word with the contextual information of each first candidate word to obtain a first fused text of each first candidate word;

[0028] Fusing each second candidate word with the contextual information of the first candidate word corresponding to each second candidate word to obtain a second fused text of each second candidate word;

[0029] Inputting the first fused text into a pre-trained attribute discriminator, and outputting a first probability of each first candidate word through the attribute discriminator, where the first probability is used to represent the probability that each first candidate word belongs to text sentiment information;

[0030] Inputting the second fused text into the attribute discriminator, and outputting the second probability of each second candidate word through the attribute discriminator, where the second probability is used to represent the probability that each second candidate word belongs to the text sentiment information;

[0031] A first word-level loss for the training text is determined based on the first probability, and a second word-level loss for the training text is determined based on the second probability.

[0032] In some implementation schemes of the present application, fine-tuning the parameters of the candidate prefix information is performed based on the training text and the pre-training model to obtain the fine-tuned prefix information corresponding to the text sentiment information, including:

[0033] Fusing the training text and the candidate prefix information to obtain second fused information;

[0034] Inputting the second fusion information into the pre-training model, and outputting the second word distribution information of the training text through the pre-training model;

[0035] Based on the second word distribution information, construct a second candidate word set for the training text;

[0036] Determining a first sequence loss and a second sequence loss of the training text based on the second candidate word set;

[0037] The parameters of the candidate prefix information are fine-tuned based on the first sequence loss and the second sequence loss to obtain the fine-tuned prefix information corresponding to the text sentiment information.

[0038] In some embodiments of the present application, generating a target text of the text to be processed based on the target fusion information includes:

[0039] Input the target fusion information into the pre-trained model, and output the predicted text of the text to be processed through the pre-trained model;

[0040] The predicted text and the text to be processed are fused to obtain the target text of the text to be processed.

[0041] In a second aspect, an embodiment of the present invention further provides a text generation device, comprising:

[0042] An information acquisition unit, used to acquire the text to be processed and the target prefix information of the text to be processed;

[0043] A prefix determination unit, used for fusing the text to be processed and the target prefix information to obtain target fusion information;

[0044] The text generation unit is used to generate a target text of the text to be processed based on the target fusion information.

[0045] In a third aspect, the present application further provides a computer device, the computer device comprising:

[0046] one or more processors;

[0047] Memory; and

[0048] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement any one of the text generation methods in the first aspect.

[0049] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and the computer program is loaded by a processor to execute the steps in any one of the text generation methods in the first aspect.

[0050] The beneficial effects of the present invention are as follows: the target fusion information is obtained by fusing the text to be processed and the target prefix information, and the target text of the text to be processed is generated based on the target fusion information. The target text that meets the specific emotional attribute label can be generated, so that the generated target text is more in line with the development of the plot, and the fluency, coherence and interest of the generated text are enhanced, thereby improving the immersive gaming experience of game players. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0052] Figure 1 It is a flowchart of a text generation method provided by an embodiment of the present invention;

[0053] Figure 2 is a flowchart of a specific embodiment of the text generation method provided by an embodiment of the present invention;

[0054] Figure 3 is a principle block diagram of a text generation device provided by an embodiment of the present invention;

[0055] Figure 4 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0057] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, which are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second", "third" and the like are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, features defined as "first", "second", "third", etc. may explicitly or implicitly include one or more features. In the description of the present application, "multiple" means two or more, and "several" means one or more, unless otherwise clearly and specifically defined.

[0058] In this application, the word "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described in this application as "exemplary" is not necessarily to be construed as being preferred or advantageous over other embodiments. The following description is given to enable any technician in the field to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present application can be implemented without using these specific details. In other instances, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.

[0059] It should be noted that since the method of the embodiment of the present application is executed in a computer device, the processing objects of each computer device exist in the form of data or information. For example, time is actually time information. It can be understood that if size, quantity, position, etc. are mentioned in subsequent embodiments, they are all corresponding data for processing by the computer device. The details will not be repeated here.

[0060] The background of the game affects the development of the plot. For example, when the background of the game is set to comedy, the plot of the game should be positive; otherwise, it should be negative. The inventor has found through research that the existing text generation method based on artificial intelligence (AI) usually generates fluent text content that meets the most basic logical requirements based on pre-trained language models and pre-learned knowledge. The text generated in this way often does not have emotional attributes and is not consistent with the development of the game plot, which weakens the fluency, coherence and fun of the game and reduces the immersive gaming experience of game players.

[0061] Based on this, in an embodiment of the present application, the text to be processed and the target prefix information of the text to be processed are obtained; the text to be processed and the target prefix information are fused to obtain target fusion information; based on the target fusion information, the target text of the text to be processed is generated, and a target text that conforms to the characteristic emotional attribute label can be generated, so that the generated target text is more in line with the development of the plot, the fluency, coherence and fun of the generated text are enhanced, and the immersive gaming experience of game players is improved.

[0062] The content of this application is further illustrated below through the description of embodiments in conjunction with the accompanying drawings.

[0063] This embodiment provides a text generation method, such as Figure 1 As shown in , the method includes:

[0064] Step S10: Obtain the text to be processed and the target prefix information of the text to be processed.

[0065] The text to be processed is the prompt text of the text generation task. For example, according to the text X 1:X ={x1,x2,…,x r}Generate continuous text X r+1:n ={x r+1 ,x r+2 ,…,x n}, then text X 1:X The text to be processed can be the first sentence, title or introductory sentence of a plot game story. The text to be recognized can be a sentence or a paragraph input by a user through a computer device (e.g., a smart phone), or text data obtained by converting speech to text of collected user conversations through a computer device. For example, the text to be processed can be "The book is".

[0066] The target prefix information is a continuous sentiment attribute vector, which is pre-set in the activation layer of the pre-trained model and is used to control the pre-trained model to generate target text belonging to the target sentiment information in the text generation task. The target prefix information can be expressed as: Where P represents the target prefix information, d emb represents the dimension of word embedding in the pre-trained model, and l represents the length of the target prefix information.

[0067] Before executing the text generation task, this embodiment generates prefix information corresponding to different sentiment attributes, for example, the sentiment attribute of "positive" or "Positive" corresponds to a positive prefix, and the sentiment attribute of "negative" or "Negative" corresponds to a negative prefix. When generating a "positive" or "positive" target text, the text to be processed and the positive prefix are obtained, and when generating a "negative" or "negative" target text, the text to be processed and the negative prefix are obtained.

[0068] In a specific implementation, obtaining target prefix information of the text to be processed in step S10 includes:

[0069] Step S11, obtaining text material information and a pre-trained model;

[0070] Step S12: determining emotion-related information based on the text material information and the pre-training model; the emotion-related information represents the corresponding relationship between the emotion information and the prefix information;

[0071] Step S13: Obtain target emotion information of the target text, and determine target prefix information of the text to be processed based on the target emotion information and emotion association information.

[0072] The sentiment association information represents the correspondence between the sentiment information and the prefix information. The text material information is a pre-set text data set used to determine the sentiment association information. Different sentiment attributes correspond to different prefix information. For example, the sentiment attribute of "positive" or "Positive" corresponds to a positive prefix, and the sentiment attribute of "negative" or "Negative" corresponds to a negative prefix.

[0073] The pre-trained model is a pre-trained language model (Pre-trained Language Models, PLMs). The pre-trained model is a model that has been trained. The sentiment association information is determined based on the text material information and the pre-trained model, so that the pre-trained model can better match downstream tasks (such as generating text with specific sentiment attributes). For example, the pre-trained model can adopt a generative pre-trained Transformer (Generative Pre-Trained Transformer, GPT) model, such as the GPT-3 model.

[0074] The target sentiment information is the sentiment attribute category of the target text that needs to be generated by the text generation task. For example, if the target sentiment information is "positive" or "Positive", it means that the text generation task needs to generate text with positive sentiment attribute category; if the target sentiment information is "negative" or "Negative", it means that the text generation task needs to generate text with negative sentiment attribute category.

[0075] Before executing the text generation task, this embodiment obtains text material information and a pre-trained model, and determines the sentiment association information based on the text material information and the pre-trained model. When executing the text generation task, the target prefix information of the text to be processed can be directly obtained based on the target sentiment information and the sentiment association information. For example, when the target sentiment information is "positive" or "Positive", the target prefix information of the text to be processed is determined to be a positive prefix; when the target sentiment information is "negative" or "Negative", the target prefix information of the text to be processed is determined to be a negative prefix.

[0076] In a specific implementation, the text material information includes the training text and the text emotion information of the training text, and step S12 includes:

[0077] Step S121, constructing initial prefix information based on text sentiment information;

[0078] Step S122: updating the parameters of the initial prefix information based on the training text and the pre-training model to obtain candidate prefix information corresponding to the text sentiment information;

[0079] Step S123, fine-tuning the parameters of the candidate prefix information based on the training text and the pre-training model to obtain the fine-tuned prefix information corresponding to the text sentiment information;

[0080] Step S124: Determine the emotion-related information based on the text emotion information and the fine-tuning prefix information.

[0081] The text material information includes the training text and the text sentiment information of the training text. The text sentiment information is the sentiment attribute category to which the training text belongs. The initial prefix information is the prefix information constructed based on the text sentiment information. The candidate prefix information is the updated prefix information obtained by updating the parameters of the initial prefix information based on the training text and the pre-trained model. The fine-tuned prefix information is the fine-tuned prefix information obtained by fine-tuning the parameters of the candidate prefix information based on the training text and the pre-trained model.

[0082] When this embodiment determines the emotion association information based on the text material information and the pre-trained model, the prefix information is first initialized based on the text emotion information of the training text to obtain the initial prefix information, and then the parameters of the initial prefix information are updated based on the training text and the pre-trained model to obtain the candidate prefix information corresponding to the text emotion information, and then the parameters of the candidate prefix information are fine-tuned based on the training text and the pre-trained model to obtain the fine-tuned prefix information corresponding to the text emotion information, and finally the emotion association information is determined based on the text emotion information and the fine-tuned prefix information. After obtaining the candidate prefix information, the parameters of the candidate prefix information are fine-tuned based on the training text and the pre-trained model, so that the pre-trained model can take the context into account when predicting the text, and ensure the controllability and fluency of the generated text sequence.

[0083] In a specific implementation, step S122 includes:

[0084] Step S1221, fusing the training text and the initial prefix information to obtain first fused information;

[0085] Step S1222: input the first fusion information into the pre-training model, and output the first word distribution information of the training text through the pre-training model;

[0086] Step S1223: construct a first candidate word set of the training text based on the first word distribution information;

[0087] Step S1224, determining a first word-level loss and a second word-level loss of the training text based on the first candidate word set;

[0088] Step S1225: Update the parameters of the initial prefix information based on the first word-level loss and the second word-level loss to obtain candidate prefix information corresponding to the text sentiment information.

[0089] The first fusion information is the fusion text obtained by fusion of the training text and the initial prefix information. The fusion process of the training text and the initial prefix information can be expressed as: X1 = concat(P1, X n ), where X1 represents the first fusion information, P1 represents the initial prefix information, and X n Represents the training text.

[0090] The first word distribution information is the distribution information of the word predicted by the pre-training model at time step t. The initial prefix information is used as the sentiment attribute prefix. The training text is modeled with the pre-training model with parameter θ. The probability distribution of the obtained training text can be expressed as: p θ1 (x 1:n ) represents the training text x 1:n The probability distribution of θ (x t|x ＜t ,P1) represents the word distribution information predicted by the pre-training model at time step t, p θ (x t |x ＜t ,P1)=softmax(h t ), represents the original output of the pre-trained model at time step t. Therefore, the probability distribution of the training text output by the pre-trained model can determine the distribution information of the words predicted by the pre-trained model at time step t.

[0091] The first candidate word set is a word set consisting of the k words with the highest probability determined based on the first word distribution information. For example, the word distribution information at time step t is the probabilities of the words good, fun, the, boring, which are 0.9, 0.7, 0.45, and 0.3 respectively. The first candidate word set is a word set consisting of the three words with the highest probability in the word distribution information. Then the first candidate word set at time step t is {good, fun, the}.

[0092] The first word-level loss is the word-level likelihood loss of the first candidate word set, which is used to characterize the probability that each word in the first candidate word set belongs to the text sentiment information, and the second word-level loss is the word-level non-likelihood loss of the first candidate word set, which is used to characterize the probability that each word in the first candidate word set does not belong to the text sentiment information. After constructing the first candidate word set for the training text, this embodiment determines the first word-level loss and the second word-level loss of the training text based on the first candidate word set, and updates the parameters of the initial prefix information based on the first word-level loss and the second word-level loss, so that the pre-training model can generate words with expected sentiment attributes as much as possible and stay away from words with undesired sentiment attributes.

[0093] In a specific implementation, the step of updating the parameters of the initial prefix information based on the first word-level loss and the second word-level loss specifically includes: determining a first target loss value based on the first word-level loss and the second word-level loss; when the first target loss value does not meet the preset first condition, updating the parameters of the initial prefix information, and continuing to perform the step of fusing the training text and the initial prefix information to obtain the first fusion information until the first target loss value meets the first condition. The calculation formula of the first target loss value is: represents the first target loss value, represents the first word level loss, represents the second word-level loss, θ represents the model parameters of the pre-trained model, represents the training text set, t represents the time step, Represents the training text set The tth time step of the ith training text in , n represents the length of the training text, and γ represents the preset ratio value.

[0094] In a specific implementation, the first candidate word set includes multiple first candidate words, and step S1224 includes:

[0095] Step S12241, constructing reverse prompt information for each first candidate word;

[0096] Step S12242: based on the reverse prompt information, determine a plurality of second candidate words corresponding to the plurality of first candidate words respectively, each second candidate word being an antonym of the first candidate word corresponding to each second candidate word;

[0097] Step S12243, fusing each first candidate word with the contextual information of each first candidate word to obtain a first fused text of each first candidate word;

[0098] Step S12244, fusing each second candidate word with the contextual information of the first candidate word corresponding to each second candidate word to obtain a second fused text of each second candidate word;

[0099] Step S12245: input the first fused text into a pre-trained attribute discriminator, and output a first probability of each first candidate word through the attribute discriminator, where the first probability is used to characterize the probability that each first candidate word belongs to text sentiment information;

[0100] Step S12246: input the second fused text into an attribute discriminator, and output a second probability of each second candidate word through the attribute discriminator, where the second probability is used to characterize the probability that each second candidate word belongs to text sentiment information;

[0101] Step S12247: determine a first word-level loss of the training text based on the first probability, and determine a second word-level loss of the training text based on the second probability.

[0102] The reverse prompt information can be expressed as: is opposite from x 1:t-1 [mask], represents the first fused text at time step t, x 1:t-1 Represents the context information at time step t. For example, the first candidate word at time step t is "good", and the context information at time step t is "The Book is", then the prompt information is The Book is good is opposite from The Book is [mask].

[0103] The first candidate word set includes multiple first candidate words, and multiple second candidate words correspond to the multiple first candidate words respectively, and each second candidate word is the antonym of the first candidate word corresponding to each second candidate word. For example, the first candidate word set is {good, fun, the, boring}, then the second candidate word set constructed based on the first candidate word set is {bad, boring, the, goood}.

[0104] In order to determine the word-level non-likelihood loss of the first candidate word set, this embodiment first constructs reverse prompt information for each first candidate word, and then inputs the reverse prompt information into a pre-trained model, and outputs the antonym (i.e., the second candidate word) corresponding to each first candidate word through the pre-trained model, thereby obtaining a third candidate word set consisting of multiple second candidate words.

[0105] The above context information is the sequence text before the first candidate word in the training text. For example, if the first candidate word corresponds to time step t, then the above context information of the first candidate word is the sequence text x before time step t in the training text. 1:t-1 The first fused text is a fused text obtained by fusing each first candidate word with the context information of each first candidate word. For example, if the first candidate word at time step t is "good" and the context information at time step t is "The Book is", then the first fused text is "The Book is good".

[0106] The second fused text is a fused text obtained by fusing each second candidate word with the contextual information of the first candidate word corresponding to each second candidate word. For example, the contextual information of the first candidate word "good" is "The Book is", and the second candidate word corresponding to the first candidate word "good" is "bad", then the second fused text is "The Book is bad".

[0107] The first probability is used to characterize the probability that each first candidate word belongs to the text sentiment information, and the second probability is used to characterize the probability that each second candidate word belongs to the text sentiment information. The attribute discriminator is a pre-trained discriminant model for outputting the first probability based on the first fused text and the second probability based on the second fused text. The first probability can be expressed as: The second probability can be expressed as: p(k) expresses the first probability of the first candidate word k, Indicates the second candidate word The second probability, represents the first fused text of the first candidate word k, Indicates the second candidate word The second fused text, a represents the text sentiment information.

[0108] For example, Figure 2 As shown, each first candidate word in the first candidate word set {good, fun, the, boring} is respectively combined with the above information x 1:t-1 After fusion, the attribute discriminator is input, and the probability of {good, fun, the, boring} belonging to positive sentiment is 0.97 / 0.85 / 0.44 / 0.10 respectively; each second candidate word {bad, boring, the, good} in the second candidate word set is respectively compared with the above information x 1:t-1 After fusion, the attributes are input into the attribute discriminator, and the output of the attribute discriminator has the probability of {bad, boring, the, good} belonging to positive emotions, which are 0.92 / 0.75 / 0.24 / 0.10 respectively.

[0109] In a specific implementation, before inputting the first fused text into a pre-trained attribute discriminator, the process includes: obtaining a training data set, the training data set including training data and true attribute labels (for example, positive or negative) corresponding to the training data; inputting the training data into the attribute discriminator, and outputting the predicted probability that the training data belongs to the true attribute label through the attribute discriminator; and training the attribute discriminator based on the predicted probability that the training text belongs to the true attribute label to obtain a pre-trained attribute discriminator.

[0110] In a specific implementation, the step of determining the first word-level loss of the training text based on the first probability specifically includes: normalizing the first probability to obtain a normalized first probability; and determining the first word-level loss based on the normalized first probability and the word distribution probability of the first candidate word. The normalized first probability can be expressed as: The calculation formula of the first word level loss is: represents the first word-level loss, p′(k) represents the normalized first probability of the first candidate word k, p(k) expresses the first probability of the first candidate word k, α represents the preset parameter, and p θ (k|x ＜t ,P) represents the word distribution probability of the first candidate word k, which can be obtained from the first word distribution information, Represents the first candidate word set.

[0111] In a specific implementation, the step of determining the second word-level loss of the training text based on the second probability specifically includes: normalizing the second probability to obtain a normalized second probability; and determining the first word-level loss based on the normalized second probability and the word distribution probability of the first candidate word corresponding to the second candidate word. The normalized second probability can be expressed as: The calculation formula for the second word-level loss is: represents the second word level loss, Indicates the second candidate word The normalized second probability, Indicates the second candidate word The second probability, p θ (k|x ＜t ,P) represents the second candidate word The word distribution probability of the corresponding first candidate word k can be obtained from the first word distribution information, Represents a third candidate word set consisting of multiple second candidate words.

[0112] In a specific implementation, step S123 includes:

[0113] Step S1231, fusing the training text and the candidate prefix information to obtain second fused information;

[0114] Step S1232: input the second fusion information into the pre-training model, and output the second word distribution information of the training text through the pre-training model;

[0115] Step S1233: construct a second candidate word set for the training text based on the second word distribution information;

[0116] Step S1234, determining a first sequence loss and a second sequence loss of the training text based on the second candidate word set;

[0117] Step S1235: fine-tune the parameters of the candidate prefix information based on the first sequence loss and the second sequence loss to obtain the fine-tuned prefix information corresponding to the text sentiment information.

[0118] Similar to the first fusion information, the second fusion information is a fusion text obtained by fusing the training text and the candidate prefix information. The process of fusing the training text and the candidate prefix information can be expressed as: X2 = concat(P2, X n ), where X2 represents the second fusion information and P2 represents the candidate prefix information.

[0119] The second word distribution information is the distribution information of the text sequence predicted by the pre-training model at time step t. The candidate prefix information is used as the sentiment attribute prefix. The training text is modeled with the pre-training model with parameter θ. The probability distribution of the obtained training text can be expressed as: p θ2 (x 1:n ) represents the training text x 1:n The probability distribution of θ (x t |x ＜t,P2) represents the word distribution information predicted by the pre-training model at time step t.

[0120] The second candidate word set is a word set constructed based on the distribution information of the text sequence predicted by the pre-training model at time step t. Since the distribution information of the text sequence includes the word distribution information of multiple words, the second candidate word set constructed based on the distribution information of the text sequence includes multiple candidate subsets, and each candidate subset is a set of k words with the highest probability in the word distribution information of each word. For example, for the continuous text X t+1:t+N ={x t+1 ,x t+2 ,…,x t+N}, and its corresponding second candidate word set is For x t+1 The set of k words with the highest probability in the word distribution information, For x t+2 The set of k words with the highest probability in the word distribution information, and so on.

[0121] The first sequence loss is the sequence-level likelihood loss of the text to be processed, which is used to characterize the continuous text X predicted by the pre-training model. t+1:t+N The step of determining the first sequence loss based on the second candidate word set includes: determining the third probability of each third candidate word in the second candidate word set, the third probability is used to characterize the probability that each third candidate word belongs to the text sentiment information; normalizing the third probability to obtain the normalized third probability; and determining the first sequence loss based on the normalized third probability and the word distribution probability of each third candidate word. The calculation formula of the first sequence loss is: represents the first sequence loss, represents the i-th candidate subset in the second candidate word set, p′[s] represents the normalized third probability of the third candidate word s in the i-th candidate subset, and p θ (s|x ＜t ,P) represents the word distribution probability of the third candidate word s in the i-th candidate subset, and θ represents the model parameters of the pre-trained model.

[0122] It should be noted that the step of determining the third probability of each third candidate word in the second candidate word set is the same as the step of determining the first probability of the aforementioned first candidate word, and the step of normalizing the third probability is the same as the step of normalizing the aforementioned first probability. For details, please refer to the steps of determining and normalizing the first probability of the aforementioned first candidate word, and this application will not repeat them here.

[0123] The second sequence loss is the sequence-level non-likelihood loss of the text to be processed, which is used to characterize the continuous text X predicted by the pre-training model.t+1:t+N The probability of not belonging to the text sentiment information, the step of determining the second sequence loss based on the second candidate word set specifically includes: constructing a fourth candidate word set based on the second candidate word set, the fourth candidate word set being the antonym set of the second candidate word set; determining the fourth probability of each fourth candidate word in the fourth candidate word set, the fourth probability being used to characterize the probability that each fourth candidate word belongs to the text sentiment information; normalizing the fourth probability to obtain the normalized fourth probability; determining the second sequence loss based on the normalized fourth probability and the word distribution probability of each third candidate word. Among them, the calculation formula of the second sequence loss is: represents the second order loss, represents the i-th candidate subset in the fourth candidate word set, Indicates the fourth candidate word The normalized fourth probability, p θ (s|x ＜t ,P) represents the word distribution probability of the third candidate word s in the i-th candidate subset, and θ represents the model parameters of the pre-trained model.

[0124] It should be noted that the step of constructing the fourth candidate word set based on the second candidate word set is the same as the step of constructing the third candidate word set based on the first candidate word set, and the specific steps can refer to the steps of constructing the third candidate word set based on the first candidate word set. In addition, the steps of determining the fourth probability and normalization of each fourth candidate word in the fourth candidate word set are the same as the steps of determining the second probability and normalization of each second candidate word, and the specific steps can refer to the steps of determining the second probability and normalization of each second candidate word, and the present application will not repeat them here.

[0125] In a specific implementation, the step of fine-tuning the parameters of the candidate prefix information based on the first sequence loss and the second sequence loss specifically includes: determining the second target loss value based on the first sequence loss and the second sequence loss; when the second target loss value does not meet the preset second condition, fine-tuning the parameters of the candidate prefix information, and continuing to perform the step of fusing the training text and the candidate prefix information to obtain the second fusion information until the second target loss value meets the second condition. The calculation formula of the second target loss value is: represents the second target loss value, represents the first sequence loss, represents the second sequence loss, θ represents the model parameters of the pre-trained model, represents the training text set, t represents the time step, Represents the training text set The tth time step of the ith training text in , n represents the length of the training text, and τ represents the preset ratio value.

[0126] Step S20: Fusing the text to be processed and the target prefix information to obtain target fusion information.

[0127] The target fusion information is the fusion text obtained by fusion of the to-be-processed text and the target prefix information. The fusion process of the to-be-processed text and the target prefix information can be expressed as: X3 = concat(P, X 1:r ), where X3 represents the target fusion information, P represents the target prefix information, and X 1:r Represents text to be processed.

[0128] Step S30: Generate a target text of the text to be processed based on the target fusion information.

[0129] The target text is a plot description of the text to be processed generated based on the target fusion information. For example, the text to be processed is X 1:r ={x1,x2,…,x r}, the target text is X 1:n ={x1,x2,…,x r ,x r+1 ,x r+2 ,…,x n This embodiment generates a target text of the text to be processed based on the target fusion information, and can generate text with specific emotional attributes, so that the generated target text is more consistent with the development of the game plot, enhances the fluency, coherence and fun of the game, and improves the immersive game experience of game players.

[0130] In a specific implementation, step S30 includes:

[0131] Step S31, inputting the target fusion information into the pre-training model, and outputting the predicted text of the text to be processed through the pre-training model;

[0132] Step S32: Fusing the predicted text with the text to be processed to obtain a target text of the text to be processed.

[0133] After the target fusion information is input into the pre-trained model, the pre-trained model will output the predicted text of the text to be processed based on the target fusion information, where the predicted text can be expressed as, X r+1:n ={x r+1 ,x r+2 ,…,x n After obtaining the predicted text, the text to be processed and the predicted text are fused to obtain the target text X that meets the target emotional information. 1:n ={x1,x2,…,x r ,xr+1 ,x r+2 ,…,x n}.

[0134] In order to better implement the text generation method in the embodiment of the present application, based on the text generation method, the embodiment of the present application also provides a text generation device, such as Figure 3 As shown, the text generation device includes:

[0135] An information acquisition unit 610 is used to acquire the text to be processed and the target prefix information of the text to be processed;

[0136] An information fusion unit 620 is used to fuse the text to be processed and the target prefix information to obtain target fusion information;

[0137] The text generation unit 630 is used to generate a target text of the text to be processed based on the target fusion information.

[0138] In an embodiment of the present application, the text to be processed and the target prefix information are fused to obtain target fusion information, and the target text of the text to be processed is generated based on the target fusion information. This can generate a target text that meets a specific emotional attribute label, so that the generated target text is more consistent with the development of the plot, thereby enhancing the fluency, coherence and fun of the generated text, and improving the immersive gaming experience of game players.

[0139] In some embodiments of the present application, the information acquisition unit 610 is specifically used to:

[0140] Obtain text material information and pre-trained models;

[0141] Based on the text material information and the pre-trained model, the sentiment-related information is determined; the sentiment-related information represents the correspondence between the sentiment information and the prefix information;

[0142] The target sentiment information of the target text is obtained, and the target prefix information of the text to be processed is determined based on the target sentiment information and the sentiment association information.

[0143] In some embodiments of the present application, the text material information includes the training text and the text emotion information of the training text, and the information acquisition unit 610 is further used to:

[0144] Based on the text sentiment information, construct the initial prefix information;

[0145] Based on the training text and the pre-training model, the parameters of the initial prefix information are updated to obtain candidate prefix information corresponding to the text sentiment information;

[0146] Fine-tune the parameters of the candidate prefix information based on the training text and the pre-training model to obtain the fine-tuned prefix information corresponding to the text sentiment information;

[0147] Based on the text sentiment information and fine-tuning prefix information, the sentiment association information is determined.

[0148] In some embodiments of the present application, the information acquisition unit 610 is further configured to:

[0149] Fusing the training text and the initial prefix information to obtain first fused information;

[0150] Inputting the first fusion information into a pre-training model, and outputting first word distribution information of the training text through the pre-training model;

[0151] Based on the first word distribution information, construct a first candidate word set for the training text;

[0152] Determine a first word-level loss and a second word-level loss of the training text based on the first candidate word set;

[0153] The parameters of the initial prefix information are updated based on the first word-level loss and the second word-level loss to obtain candidate prefix information corresponding to the text sentiment information.

[0154] In some embodiments of the present application, the first candidate word set includes multiple first candidate words, and the information acquisition unit 610 is further configured to:

[0155] Construct reverse prompt information for each first candidate word;

[0156] Based on the reverse prompt information, determining a plurality of second candidate words corresponding to the plurality of first candidate words respectively, each second candidate word being an antonym of the first candidate word corresponding to each second candidate word;

[0157] Fusing each first candidate word with the contextual information of each first candidate word to obtain a first fused text of each first candidate word;

[0158] Merging each second candidate word with the context information of the first candidate word corresponding to each second candidate word to obtain a second fused text of each second candidate word;

[0159] Inputting the first fused text into a pre-trained attribute discriminator, and outputting a first probability of each first candidate word through the attribute discriminator, where the first probability is used to represent the probability that each first candidate word belongs to text sentiment information;

[0160] Inputting the second fused text into the attribute discriminator, and outputting the second probability of each second candidate word through the attribute discriminator, where the second probability is used to represent the probability that each second candidate word belongs to the text sentiment information;

[0161] A first word-level loss for the training text is determined based on the first probability, and a second word-level loss for the training text is determined based on the second probability.

[0162] In some embodiments of the present application, the information acquisition unit 610 is further configured to:

[0163] Fusing the training text and the candidate prefix information to obtain second fused information;

[0164] Inputting the second fusion information into the pre-training model, and outputting the second word distribution information of the training text through the pre-training model;

[0165] Based on the second word distribution information, construct a second candidate word set for the training text;

[0166] Determining a first sequence loss and a second sequence loss of the training text based on the second candidate word set;

[0167] The parameters of the candidate prefix information are fine-tuned based on the first sequence loss and the second sequence loss to obtain the fine-tuned prefix information corresponding to the text sentiment information.

[0168] In some embodiments of the present application, the text generation unit 630 is specifically used to:

[0169] Input the target fusion information into the pre-trained model, and output the predicted text of the text to be processed through the pre-trained model;

[0170] The predicted text and the text to be processed are fused to obtain the target text of the text to be processed.

[0171] The embodiment of the present application further provides a computer device, which integrates any text generation device provided in the embodiment of the present application, and the computer device includes:

[0172] one or more processors;

[0173] Memory; and

[0174] One or more applications, wherein the one or more applications are stored in a memory and are configured to execute, by a processor, the steps of the text generation method in any of the above-mentioned text generation method embodiments.

[0175] The present application also provides a computer device that integrates any of the text generation devices provided in the present application. Figure 4 As shown, it shows a schematic diagram of the structure of the computer device involved in the embodiment of the present application, specifically:

[0176] The computer device may include one or more processing core processors 801, one or more computer-readable storage media memories 802, a power supply 803, an input unit 804 and other components. Those skilled in the art will appreciate that Figure 4The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0177] The processor 801 is the control center of the computer device. It uses various interfaces and lines to connect various parts of the entire computer device. By running or executing software programs and / or modules stored in the memory 802 and calling data stored in the memory 802, it executes various functions of the computer device and processes data, thereby monitoring the computer device as a whole. Optionally, the processor 801 may include one or more processing cores; preferably, the processor 801 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 801.

[0178] The memory 802 can be used to store software programs and modules. The processor 801 executes various functional applications and data processing by running the software programs and modules stored in the memory 802. The memory 802 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 802 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 802 may also include a memory controller to provide the processor 801 with access to the memory 802.

[0179] The computer device also includes a power supply 803 for supplying power to each component. Preferably, the power supply 803 can be logically connected to the processor 801 through a power management system, so that the power management system can manage charging, discharging, power consumption and other functions. The power supply 803 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0180] The computer device may further include an input unit 804, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0181] Although not shown, the computer device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 801 in the computer device will load the executable files corresponding to the processes of one or more application programs into the memory 802 according to the following instructions, and the processor 801 will run the application programs stored in the memory 802, thereby realizing various functions, as follows:

[0182] Obtain the text to be processed and the target prefix information of the text to be processed;

[0183] Fusing the text to be processed and the target prefix information to obtain target fusion information;

[0184] Based on the target fusion information, a target text of the text to be processed is generated.

[0185] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0186] To this end, the embodiment of the present application provides a computer-readable storage medium, which may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any text generation method provided in the embodiment of the present application. For example, the computer program is loaded by a processor to execute the following steps:

[0187] Obtain the text to be processed and the target prefix information of the text to be processed;

[0188] Fusing the text to be processed and the target prefix information to obtain target fusion information;

[0189] Based on the target fusion information, a target text of the text to be processed is generated.

[0190] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the detailed description of other embodiments above, and will not be repeated here.

[0191] In specific implementation, the above units or structures can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units or structures can refer to the previous method embodiments, which will not be repeated here.

[0192] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0193] The above is a detailed introduction to a text generation method, device, computer equipment and computer-readable storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A text generation method, characterized in that: include: Obtaining a text to be processed and target prefix information of the text to be processed; Fusing the text to be processed and the target prefix information to obtain target fusion information; Based on the target fusion information, a target text of the text to be processed is generated.

2. The method according to claim 1, characterized in that: Obtaining target prefix information of the text to be processed, including: Obtain text material information and pre-trained models; Based on the text material information and the pre-trained model, determining the emotion-related information; the emotion-related information represents the corresponding relationship between the emotion information and the prefix information; Target emotion information of the target text is acquired, and target prefix information of the text to be processed is determined based on the target emotion information and the emotion association information.

3. The method according to claim 2, characterized in that The text material information includes a training text and text emotion information of the training text, and the emotion-related information is determined based on the text material information and the pre-training model, including: Based on the text sentiment information, construct initial prefix information; Based on the training text and the pre-training model, the parameters of the initial prefix information are updated to obtain candidate prefix information corresponding to the text sentiment information; Fine-tune the parameters of the candidate prefix information based on the training text and the pre-training model to obtain the fine-tuned prefix information corresponding to the text sentiment information; Based on the text sentiment information and the fine-tuning prefix information, sentiment association information is determined.

4. The method according to claim 3, characterized in that The updating of the parameters of the initial prefix information based on the training text and the pre-training model to obtain candidate prefix information corresponding to the text sentiment information includes: Fusing the training text and the initial prefix information to obtain first fused information; Inputting the first fusion information into the pre-training model, and outputting first word distribution information of the training text through the pre-training model; Based on the first word distribution information, construct a first candidate word set for the training text; Determining a first word-level loss and a second word-level loss of the training text based on the first candidate word set; The parameters of the initial prefix information are updated based on the first word-level loss and the second word-level loss to obtain candidate prefix information corresponding to the text sentiment information.

5. The method according to claim 4, characterized in that The first candidate word set includes a plurality of first candidate words, and determining the first word-level loss and the second word-level loss of the training text based on the first candidate word set includes: Constructing reverse prompt information for each of the first candidate words; Based on the reverse prompt information, determining a plurality of second candidate words corresponding to the plurality of first candidate words respectively, each of the second candidate words being an antonym of the first candidate word corresponding to each of the second candidate words; fusing each of the first candidate words with the context information of each of the first candidate words to obtain a first fused text of each of the first candidate words; Merging each of the second candidate words with the context information of the first candidate word corresponding to each of the second candidate words to obtain a second fused text of each of the second candidate words; Inputting the first fused text into a pre-trained attribute discriminator, and outputting a first probability of each first candidate word through the attribute discriminator, where the first probability is used to represent the probability that each first candidate word belongs to the text sentiment information; Inputting the second fused text into the attribute discriminator, and outputting a second probability of each second candidate word through the attribute discriminator, wherein the second probability is used to represent the probability that each second candidate word belongs to the text sentiment information; A first word-level loss of the training text is determined based on the first probability, and a second word-level loss of the training text is determined based on the second probability.

6. The method according to claim 3, characterized in that: The fine-tuning of the parameters of the candidate prefix information based on the training text and the pre-training model to obtain the fine-tuned prefix information corresponding to the text sentiment information includes: Fusing the training text and the candidate prefix information to obtain second fused information; Inputting the second fusion information into the pre-training model, and outputting second word distribution information of the training text through the pre-training model; Based on the second word distribution information, construct a second candidate word set for the training text; Determining a first sequence loss and a second sequence loss of the training text based on the second candidate word set; Fine-tune the parameters of the candidate prefix information based on the first sequence loss and the second sequence loss to obtain fine-tuned prefix information corresponding to the text sentiment information.

7. The method according to claim 1, characterized in that The step of generating a target text of the text to be processed based on the target fusion information includes: Inputting the target fusion information into a pre-training model, and outputting a predicted text of the text to be processed through the pre-training model; The predicted text and the text to be processed are fused to obtain a target text of the text to be processed.

8. A text generation device, characterized in that: include: An information acquisition unit, used to acquire a text to be processed and target prefix information of the text to be processed; An information fusion unit, used for fusing the text to be processed and the target prefix information to obtain target fusion information; A text generation unit is used to generate a target text of the text to be processed based on the target fusion information.

9. A computer device, characterized in that: The computer device comprises: one or more processors; Memory; and One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the text generation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in the text generation method according to any one of claims 1 to 7.

Images