A text generator model training method

Abstract

Embodiments of the present application disclose a text generator model training method, which comprises the following steps: obtaining a data set for model training, wherein the data set comprises a plurality of annotated completed topic texts; inputting the data set into a generator model to obtain a text generated by the generator model; and training and optimizing the generator model based on the text generated by the generator model and the training texts in the data set, so that the optimized generator model can generate a text close to a topic text. The generator model trained by the above method can automatically generate a text conforming to a real topic text, thereby avoiding the problem of poor effect caused by the machine writing monotonous and repetitive release of the same opinion in related technologies.

Description

Technical Field

[0001] This application relates to the field of model training technology, and more particularly to a method for training a text generator model. Background Technology

[0002] Social networks have always been the best platform for users to obtain news and speak freely, and they are now an indispensable part of people's daily lives. However, social networks are not only filled with real and positive information, but also with false and malicious remarks. Some organizations and individuals even use social robots to continuously publish a large number of remarks with specific tendencies through multiple accounts in a short period of time on a certain event or hot topic, thereby achieving the purpose of distorting the truth and having ulterior motives. Some hostile forces even use it as a powerful means to launch information warfare. As a result, it may not only trigger the occurrence of malicious events and public opinion, but may also lead to the control and exploitation of social platforms and the loss of network control. It will not only affect the healthy development of social networks, but also destroy the harmony and stability of society [1].

[0003] In order to guide public opinion in the right direction and effectively counter the large amount of malicious information released on social networks, a large number of topic texts with positive public opinion effects can be released to social networks in a short period of time [2], thereby changing people's perception of negative public opinion and malicious public opinion events, reversing the direction of social public opinion, and winning the information war. However, relying solely on manual writing and publishing of online commentary topic texts is inefficient, a drop in the ocean, and cannot play the due role. Using machine writing to monotonously and repeatedly publish the same online commentary topic texts can only achieve half the result with twice the effort. Therefore, researching and designing an automatic generation method for online commentary topic texts based on intelligent writing technology has significant research significance and application value. Summary of the Invention

[0004] To address or partially address the aforementioned problems, this application provides a method for training a text generator model.

[0005] This application proposes a method for training a text generator model. The method includes: acquiring a dataset for model training, the dataset including multiple labeled topic texts; inputting the dataset into a generator model to obtain text generated by the generator model; and training and optimizing the generator model based on the text generated by the generator model and the training text in the dataset, so that the optimized generator model can generate text that is similar to the topic texts.

[0006] In some examples, obtaining a dataset for model training includes: obtaining an initial training set, which includes multiple obtained topic texts; filtering the topic texts in the initial training set for value orientation to obtain topic texts that conform to the current required value orientation; after labeling all topic texts that conform to the current required value orientation for sentiment orientation, segmenting all topic texts that conform to the current required value orientation for word formation to obtain the training set for model training.

[0007] In some examples, inputting the dataset into a generator model to obtain text generated by the generator model includes: inputting the dataset into the encoder of the generator model to obtain semantic vectors corresponding to each topic text in the dataset, generating a semantic vector set; inputting the semantic vector set into the decoder of the generator model to obtain a decoding vector matrix corresponding to each semantic vector in the semantic vector set; and performing dimensionality transformation on the decoding vector matrix to obtain the text generated by the generator model.

[0008] In some examples, the dataset is input into the encoder of the generator model to obtain the semantic vector corresponding to each topic text in the dataset, generating a semantic vector set, including: inputting each topic text in the dataset into the embedding layer of the encoder of the generator model to obtain word vectors corresponding to all words in each topic text, forming a word vector matrix corresponding to each topic text; inputting the word vector matrix corresponding to each topic text into a two-layer BIGRU network to perform word order feature extraction and encoding to obtain an encoding vector matrix corresponding to each topic text, and using each encoding vector matrix as the semantic vector corresponding to each topic text.

[0009] In some examples, the semantic vector set is input into the decoder of the generator model to obtain the decoded vector matrix corresponding to each semantic vector in the semantic vector set, including: feeding each semantic vector in the semantic vector set into the TTCN network of the decoder, performing a causal convolution operation on each semantic vector to obtain the convolution output corresponding to each semantic vector; and feeding the convolution output corresponding to each semantic vector into the residual connection layer to obtain the decoded vector matrix corresponding to each semantic vector.

[0010] In some examples, the decoded vector matrix is ​​dimensionally transformed to obtain the text generated by the generator model, including: using a fully connected layer to perform dimensionality transformation on the decoded vector matrix corresponding to each semantic vector to obtain the generated text corresponding to each semantic vector.

[0011] In some examples, the generator model is trained and optimized based on the text generated by the generator model and the training text in the dataset, including: obtaining the classification results of the topic text in the dataset and the classification results of the text generated by the generator model; and optimizing the parameters of the generator model based on the classification results of the topic text in the dataset and the classification results of the text generated by the generator model.

[0012] Compared with the prior art, this application has the following beneficial effects:

[0013] This application provides a method for training a text generator model. The method includes: acquiring a dataset for model training, the dataset including multiple labeled topic texts; inputting the dataset into a generator model to obtain text generated by the generator model; training and optimizing the generator model based on the text generated by the generator model and the training text in the dataset, so that the optimized generator model can generate text that is close to the topic text. The generator model trained by the above method can automatically generate text that conforms to real topic texts, avoiding the problem of poor performance caused by machine-written and repetitive online commentary topic texts with the same opinions in related technologies. Attached Figure Description

[0014] Figure 1 This is a basic schematic diagram illustrating a text generator model training method according to an embodiment of this application;

[0015] Figure 2 This is a basic schematic diagram of a T-SeqGAN model shown in an embodiment of this application;

[0016] Figure 3 This is a basic schematic diagram of an encoder structure shown in an embodiment of this application;

[0017] Figure 4 This is a basic schematic diagram of a TCN model shown in an embodiment of this application;

[0018] Figure 5 This is a basic schematic diagram of a decoder shown in an embodiment of this application;

[0019] Figure 6 This is a basic schematic diagram of a discriminator shown in an embodiment of this application. Detailed Implementation

[0020] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0021] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0022] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0023] It should also be noted that "multiple" as mentioned in this application refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0024] Example 1

[0025] like Figure 1 As shown, Figure 1 This example proposes a method for training a text generator model, the method comprising:

[0026] S101. Obtain a dataset for model training, wherein the dataset includes multiple labeled topic texts;

[0027] S102. Input the dataset into the generator model to obtain the text generated by the generator model;

[0028] S103. The generator model is trained and optimized based on the text generated by the generator model and the training text in the dataset, so that the optimized generator model can generate text that is similar to the topic text.

[0029] In some examples, obtaining a dataset for model training includes: obtaining an initial training set, which includes multiple obtained topic texts; filtering the topic texts in the initial training set for value orientation to obtain topic texts that conform to the current required value orientation; after labeling all topic texts that conform to the current required value orientation for sentiment orientation, segmenting all topic texts that conform to the current required value orientation for word formation to obtain the training set for model training.

[0030] In some examples, inputting the dataset into a generator model to obtain text generated by the generator model includes: inputting the dataset into the encoder of the generator model to obtain semantic vectors corresponding to each topic text in the dataset, generating a semantic vector set; inputting the semantic vector set into the decoder of the generator model to obtain a decoding vector matrix corresponding to each semantic vector in the semantic vector set; and performing dimensionality transformation on the decoding vector matrix to obtain the text generated by the generator model.

[0031] In some examples, the dataset is input into the encoder of the generator model to obtain the semantic vector corresponding to each topic text in the dataset, generating a semantic vector set, including: inputting each topic text in the dataset into the embedding layer of the encoder of the generator model to obtain word vectors corresponding to all words in each topic text, forming a word vector matrix corresponding to each topic text; inputting the word vector matrix corresponding to each topic text into a two-layer BIGRU network to perform word order feature extraction and encoding to obtain an encoding vector matrix corresponding to each topic text, and using each encoding vector matrix as the semantic vector corresponding to each topic text.

[0032] In some examples, the semantic vector set is input into the decoder of the generator model to obtain the decoded vector matrix corresponding to each semantic vector in the semantic vector set, including: feeding each semantic vector in the semantic vector set into the TTCN network of the decoder, performing a causal convolution operation on each semantic vector to obtain the convolution output corresponding to each semantic vector; and feeding the convolution output corresponding to each semantic vector into the residual connection layer to obtain the decoded vector matrix corresponding to each semantic vector.

[0033] In some examples, the decoded vector matrix is ​​dimensionally transformed to obtain the text generated by the generator model, including: using a fully connected layer to perform dimensionality transformation on the decoded vector matrix corresponding to each semantic vector to obtain the generated text corresponding to each semantic vector.

[0034] In some examples, the generator model is trained and optimized based on the text generated by the generator model and the training text in the dataset, including: obtaining the classification results of the topic text in the dataset and the classification results of the text generated by the generator model; and optimizing the parameters of the generator model based on the classification results of the topic text in the dataset and the classification results of the text generated by the generator model.

[0035] Specifically, the method consists of two parts:

[0036] Part 1 (1) Data Collection and Preprocessing: Taking the collection of online topic data as an example, the initial dataset for the experiment is all online content with the topic "I support basketball" (e.g., Weibo content, Baidu hot search content). By preprocessing the initial dataset, the final dataset used for model training is obtained.

[0037] Part 2 (2) Text Generator Model T-SeqGAN: For the pre-processed model-trained dataset, firstly, the encoder part of its generator model, with BiGRU as the backbone network, extracts the word order features of the topic text in the dataset and encodes the topic text into semantic vectors; then, the decoder part, with TCN as the backbone network, extracts the global and deep semantic features of the topic text, and decodes the semantic vectors into generated text under the guidance of the above topic text features; further, the generated text is fed into the discriminator model with TCN+Attention structure as the backbone network, and the quality of the generated text is evaluated based on the semantic and grammatical correctness of the generated text and the fluency of the topic text. The evaluation results are fed back to the generator model, allowing the generator model to optimize the model based on the evaluation results.

[0038] Specifically, in the first step,

[0039] As mentioned earlier, to achieve automatic generation of text (online commentary topic text), we use automated tools (such as web crawlers) to obtain all online content under the topic "I support basketball" as the initial dataset. Furthermore, we preprocess the initial dataset using the following steps to obtain the final dataset used for model training:

[0040] 1. Delete topic text that is irrelevant to the topic;

[0041] 2. Remove English and non-textual symbols from the topic text, such as punctuation and emoticons;

[0042] 3. Based on the content of the topic text, label each topic text with its value orientation: topic texts with positive values ​​are labeled "1", and topic texts with negative or neutral values ​​are labeled "0". At this point, select the content to save according to the needs. For example, if the requirement is to generate online commentary topic texts with positive value orientation, then in this step, the topic texts labeled "0" will be temporarily saved, and the topic texts labeled "1" will be selected for further processing.

[0043] 4. Based on the content of the topic text, label the topic text with sentiment, marking positive sentiment topic text as "1" and negative sentiment topic text as "0";

[0044] 5. Perform word segmentation on the topic text after sentiment annotation, and delete redundant words in the topic text that exceed the preset maximum sentence length max_sentence_length.

[0045] After the above processing, we obtain topic texts with positive sentiment and topic texts with negative sentiment. The obtained topic texts with positive and negative sentiment are then fed into T-SeqGAN for model training. For example, we will use 1463 topic texts with positive sentiment and 691 topic texts with negative sentiment as an example for further explanation.

[0046] Model Structure

[0047] To address the issues of grammatical correctness, sentence fluency, and semantic similarity to real-world online comment text generated using existing methods, we propose an automatic online comment text generation model—Temporal Sequence Generative Adversarial Networks (T-SeqGAN). The T-SeqGAN model structure is as follows: Figure 2 As shown, it is mainly divided into generator model and discriminator model. The generator model is further divided into encoder part and decoder part.

[0048] Generator Model Design

[0049] To achieve higher grammatical correctness and greater semantic similarity to real topic texts, we improved the SeqGAN generator by adopting a Seq2Seq structure.

[0050] The encoder part of the generator model consists of one embedding layer and one two-layer BiGRU network, and its specific structure is as follows: Figure 3 As shown. Figure 3 In the code, the encoder input x i(i∈{1,2,...,n})x i (i∈{1,2,...,n}) represents the topic text, and the output is c. i (i∈{1,2,...,n})c i (i∈{1,2,...,n}) is the semantic vector of the i-th topic text.

[0051] During training, the encoder receives topic text x = {x1, x2, ..., x...} n}(where x i Given a topic text containing m words (i = {1, 2, ..., n}), the received topic text is fed to the Embedding layer. The Embedding layer converts the words in the topic text into word vectors, forming a word vector matrix for the topic text. Next, the word vector matrix is ​​input into a two-layer BiGRU network for word order feature extraction and encoding, ultimately obtaining the encoded vector matrix W. Enc-out This is the output of the encoder. The calculation of the above process in each GRU unit at time t is shown in equations (1)-(4):

[0052] z t =σ(W z [h t-1 ,x t (1)

[0053] r t =σ(W r [h t-1 ,x t (2)

[0054]

[0055]

[0056] Where, x t Given the input at time t, h t-1 For t t-1 Output at time, W z To update the gate weights, W r To reset the gate weights. W represents the weights calculated during the forward propagation of the GRU cell. The intermediate state output calculated for it is the Hadamard product operation, h t This is the final semantic vector output. The encoding vector matrix W is obtained by concatenating the outputs of each GRU unit. Enc-out .

[0057] The encoder uses a BiGRU network to efficiently extract word order features from topic texts, which can provide guidance for the decoding process. This allows the decoder to fully consider the relative rationality of each word's position in the topic text and the impact of the word's position on the semantics of the topic text. As a result, the decoder can decode texts with more reasonable word order, that is, generate texts with higher grammatical correctness.

[0058] The following issues need to be considered during the design process of the decoder section:

[0059] When RNN and its variants are used as the backbone network of the decoder, they can effectively extract global semantic features related to temporal features. However, as mentioned above, this type of model lacks the ability to extract deep semantic features of real text. However, CNN network, which is also a representative model in the field of NLP, has outstanding performance in extracting deep semantic features of topic text

[20] . However, due to the poor ability of CNN to extract and process temporal features, this model is difficult to accurately extract global semantic features of real topic text.

[0060] To integrate the advantages of RNNs and CNNs while mitigating their disadvantages, this example employs a Temporal Convolutional Neural Network (TCN) based on a dilated causal fully convolutional network. Its main structure is as follows: Figure 4 As shown, the model mainly consists of causal convolutional modules with added dilation factors. These causal convolutional modules retain the deep feature extraction capabilities of convolutional neural networks while imposing strict temporal constraints on the convolution results. This ensures that historical information is not forgotten and that the model's data processing maintains strict temporal characteristics. The addition of the dilation factor increases the receptive field of the sliding window, allowing the model to obtain information over a longer historical period using fewer network layers, thus improving training efficiency. Furthermore, TCN introduces residual modules into the network, effectively preventing gradient vanishing and gradient exploding problems that may occur during training due to increased network layer depth.

[0061] Based on the superior performance of the TCN network, in order to better extract the global and deep semantic features of real text, this patent introduces TCN into the decoder part, making it the backbone network of the decoder.

[0062] The decoder consists of a three-layer TCN network and a fully connected output layer, as shown in the following structure: Figure 5 As shown. The input to the decoder is the semantic vector c obtained through encoding. i(i∈{1,2,...,n}), the output is the decoded generated text x′ i (i∈{1,2,...,n}). The decoder feeds the received encoder output into the TCN network, which first performs a causal convolution operation on the input.

[0063] Let the input x be W Enc-out Given a sequence s with convolution kernel f:{0,...,k-1}→R, the expression for the causal convolution operation F with an added dilation factor on the elements s of that sequence is:

[0064]

[0065] Where d is the dilation factor, k is the kernel size, f(i) represents performing a convolution operation on the i-th input, and *d is the convolution operation operator.

[0066] After the convolution operation, its output is fed into the residual connection layer:

[0067] o=Activation[x+F(x)] (6)

[0068] Where F(x) is the output of the convolutional layer corresponding to the input sequence x, Activation(.) is the activation function, and o is the final output of the sequence in the TCN process.

[0069] After the above processing, W is obtained. Enc-out The decoding vector matrix W Dec-out Next, a fully connected layer is used to process W. Dec-out After dimensional transformation, the final output is the decoded text: x′={x′1,x′2,...,x′ n}

[0070] In this patented design, the superiority of the decoder lies in the fact that, like traditional CNN networks, TCN networks can extract global semantic features from input data using shallow networks and local deep semantic features using deep networks. This ensures that the topic text decoding process is based on the decoder's comprehensive and in-depth semantic understanding of the real topic text. Furthermore, compared to traditional CNN networks, TCN networks are less prone to historical information omission during decoding. That is, detailed information at each time step is fully utilized during topic text decoding. Therefore, this ensures that the decoder can utilize more detailed semantic feature information for topic text decoding, resulting in decoded text that has more similar semantic features to the real topic text.

[0071] Therefore, the generator model proposed in this patent can not only extract the contextual temporal features of real topic text, as well as global and deep semantic information, but also accurately decode the features of topic text, so that the generated text has a more similar word order and semantic structure to the real topic text.

[0072] T-SeqGAN discriminator model design

[0073] The original SeqGAN uses a CNN network as the backbone network of its discriminator model. As mentioned earlier, CNN networks are weak in extracting temporal features from generated text. Therefore, the discriminator has difficulty accurately extracting the word order, grammar, and semantic features of the generated text based on human reading order, and thus it is difficult to give an accurate evaluation of the quality of the generated text.

[0074] To accurately evaluate the quality of the generated text and provide precise guidance for generator optimization, this patent modifies the CNN-based discriminator in the original SeqGAN. The modified discriminator consists of one embedding layer, one three-layer TCN network, one attention layer, and one fully connected classification output layer, as follows: Figure 6 As shown. The input to the discriminator is the generated text and the real topic text, and the output is the probability of classifying each piece of input text as "true" or "false".

[0075] When the generated text and the real topic text are input to the discriminator, the TCN network obtains the word vector matrix output by the embedding layer. Then, it extracts the temporal and semantic features of the input topic text and outputs the discriminant vector matrix W. Dis-out W is obtained through TCN. Dis-out The calculation process is the same as in the decoder, and will not be described again.

[0076] Following the output layer of the TCN network, an Attention layer is connected. By assigning different attention weights to each word vector in the TCN network's output matrix, the importance of different words in the text can be effectively distinguished, improving the accuracy of the discriminator's classification. To dynamically adjust the weight values ​​of each word according to different semantic environments, the formula for calculating the Attention weights, attn(.), is as follows:

[0077]

[0078] Where W1 and W2 are the weight vectors of two different multilayer perceptrons, and tanh(.) is the Tanh activation function.

[0079] After obtaining the output of the Attention layer, it is input into the classification layer, causing the classification layer to... Dis-out Guided by the attention weights, the generated text is classified. This yields the discriminator's classification result for the input text. Furthermore, the generator model optimizes its parameters based on this classification result.

[0080] It is particularly important to note that, as mentioned earlier, TCN has the characteristic of being able to process data strictly according to time series. To improve the fluency of the generated text, the discriminator model needs to extract the word order and semantic features of the generated text according to human unidirectional reading habits, and further determine whether the aforementioned features of the generated text are consistent with the linguistic features of real online commentary texts. Therefore, this patent utilizes the strict time constraint property of TCN to design a unidirectional network structure for the discriminator. This allows for the unidirectional and sequential extraction and analysis of word order, grammar, and semantic features of the generated text, and further, based on the classification results of the output layer, guides the optimization of the generator model, so that the text generated by the model has higher fluency, grammatical correctness, and higher semantic similarity to real topic texts.

[0081] Model parameter settings

[0082] The model described in this application was built in the PyTorch environment. In the encoder of the T-SeqGAN generator, the number of neurons in both the hidden and output layers of the BiGRU is 32. In the decoder of the T-SeqGAN generator, the TCN has a kernel size of 3, 32 convolutional channels, and a dilation factor of 2. The output layer of the decoder (i.e., the output layer of the generator) is a 1-layer fully connected network with the same number of neurons as the number of words in the corpus dictionary constructed by the model.

[0083] The TCN parameters in the discriminator are set the same as those in the decoder, and the number of neurons in the hidden layer of the Attention layer is 32. The discriminator is a binary classifier, and the corresponding number of neurons in the output layer is 2.

[0084] To enable the generator model in the T-SeqGAN model to generate compliant positive and negative online comment texts, it requires two training steps: pre-training and adversarial training. Pre-training includes generator pre-training and discriminator pre-training, with 200 and 4 pre-training iterations respectively. Based on experimental results, the adversarial training is set to 100 epochs, with one generator training iteration and four discriminator training iterations per epoch. This patent sets the batch size to 10 and the max_sentence_length to 40.

[0085] During pre-training, the generator uses the loss function NLLLoss:

[0086]

[0087] M represents the number of categories, y ic It is an indicator variable with a value of 0 or 1 (1 if the classified category is the same as the sample category, 0 otherwise), p ik This represents the probability that the observed sample i belongs to category k.

[0088] The generator uses the CrossEntropyLoss loss function:

[0089]

[0090] y i To represent the label of sample i, a real sample is labeled 1, and a generated sample is labeled 0. During training, p i This represents the probability that topic text i is predicted to be a true topic text.

[0091] (1) A T-SeqGAN model, Temporal Sequence Generative Adversarial Network (T-SeqGAN), is proposed. T-SeqGAN modifies the generator model of Sequence Generative Adversarial Network (SeqGAN) into a Seq2Seq structure, using a Bidirectional Gated Recurrent Unit (BiGRU) as its encoder and a Temporal Convolutional Neural Network (TCN) as its decoder. In this way, the BiGRU-based encoder can effectively obtain the bidirectional temporal features of real topic text, while the TCN-based decoder can accurately obtain the deep semantic features of real topic text. This ensures that the text generated by the generator has a more similar word order structure and semantic features to real topic text;

[0092] (2) T-SeqGAN modifies the discriminator model of SeqGAN into a TCN+Attention structure model. In this way, TCN can not only obtain the positive temporal features and deep semantic features of the generated text, but also effectively distinguish the importance of different features by adding the Attention mechanism. This can effectively improve the discrimination efficiency, provide more accurate optimization guidance for the generator, and ultimately improve the fluency of the generated text.

[0093] The text generator model training method provided in this application aims to address the error accumulation problem during the training of the Seq2Seq model. It proposes to place the Seq2Seq model into the SeqGAN framework as the generator model of SeqGAN. By utilizing the reinforcement learning strategy and adversarial training process of SeqGAN, the Seq2Seq model is optimized, thereby accelerating the optimization speed, simplifying the optimization difficulty, and improving the fluency of the generated text.

[0094] Secondly, in order to address the problem that using RNN or its variants as decoders makes it difficult to fully extract the global semantic features of real topic texts, we propose using TCN as the backbone network of its decoder to fully extract the global and deep semantic features of real topic texts, thereby improving the semantic similarity between the generated text and the real topic text.

[0095] Furthermore, to address the issue of weak temporal feature extraction capability of the discriminator in the original SeqGAN model, which uses CNN as the backbone network, a model with a modified discriminator and TCN+Attention mechanism as the backbone network is proposed. This allows for the extraction of word order, grammar, and semantic features of the generated text based on human reading habits, enabling a more accurate evaluation of the quality of the generated text. In addition, it can provide more precise guidance for optimizing the generator model.

[0096] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0097] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention, depending on actual needs.

[0098] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0099] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0100] The technical solutions provided by the embodiments of the present invention have been described in detail above. Specific examples have been used in this patent to illustrate the principles and implementation methods of the embodiments of the present invention. The descriptions of the embodiments above are only for helping to understand the principles of the embodiments of the present invention. The above descriptions are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for training a generator model of text, characterized in that, The method comprises: obtaining a data set for model training, wherein the data set comprises a plurality of annotated topic texts; inputting the data set into a generator model to obtain text generated by the generator model; training and optimizing the generator model based on the text generated by the generator model and the training text in the data set, so that the optimized generator model can generate text close to the topic text; inputting the data set into the generator model to obtain text generated by the generator model, comprising: inputting the data set into the encoder of the generator model to obtain a semantic vector corresponding to each topic text in the data set, and generating a semantic vector set; inputting the semantic vector set into the decoder of the generator model to obtain a decoding vector matrix corresponding to each semantic vector in the semantic vector set; dimension conversion is performed on the decoding vector matrix to obtain the text generated by the generator model; inputting the data set into the encoder of the generator model to obtain a semantic vector corresponding to each topic text in the data set, and generating a semantic vector set, comprising: inputting each topic text in the data set into the embedding layer of the encoder of the generator model to obtain a word vector corresponding to all words in each topic text, forming a word vector matrix corresponding to each topic text; inputting the word vector matrix corresponding to each topic text into a two-layer BIGRU network to perform syntax feature extraction and coding, and obtaining an encoding vector matrix corresponding to each topic text, wherein each encoding vector matrix is used as a semantic vector corresponding to each topic text; inputting the semantic vector set into the decoder of the generator model to obtain a decoding vector matrix corresponding to each semantic vector in the semantic vector set, comprising: inputting each semantic vector in the semantic vector set into the TCN network of the decoder to perform causal convolution operation on each semantic vector to obtain a convolution output corresponding to each semantic vector; inputting the convolution output corresponding to each semantic vector into a residual connection layer to obtain a decoding vector matrix corresponding to each semantic vector. obtaining a data set for model training, comprising:

2. The method of claim 1, wherein, obtaining an initial training set, wherein the initial training set comprises a plurality of obtained topic texts; performing value orientation screening on the topic texts in the initial training set to obtain topic texts conforming to the current required value orientation; performing sentiment orientation labeling on all topic texts conforming to the current required value orientation, and then performing word segmentation on all topic texts conforming to the current required value orientation to obtain the training set for model training. dimension conversion is performed on the decoding vector matrix to obtain the text generated by the generator model, comprising:

3. The method of claim 1, wherein, dimension conversion is performed on the decoding vector matrix to obtain the text generated by the generator model, comprising: dimension conversion is performed on the decoding vector matrix to obtain the text generated by the generator model, comprising:

4. The method of claim 1, wherein, obtaining the classification results of the topic texts in the data set and the classification results of the text generated by the generator model; ​ performing parameter optimization on the generator model based on the classification results of the topic text in the data set and the classification results of the text generated by the generator model.

Images