Text generator training method and text generator training system

Abstract

The present invention provides a text generator training method and a text generator training system, wherein the method comprises: performing random Gaussian initialization on a hidden layer state vector; reading the hidden layer state vector and a starting character as the first input of the text generator; generating the first character by a polynomial sampling method as the first output of the text generator; using the character whose generation probability meets a preset condition as the next output of the text generator until an end character is generated. In the present invention, by performing random Gaussian initialization on the hidden layer state vector, it is possible to effectively avoid the text generator from generating repeated texts, thereby improving the diversity of the generated samples.

Description

Technical Field

[0001] The present invention relates to the field of Internet technology, and in particular to a text generator training method and a text generator training system. Background Art

[0002] Sample generation technology refers to the technology that uses certain technical means to discover the distribution pattern of existing samples and generate new samples that conform to their distribution. In specific scenarios such as text content risk control, it is necessary to discover cheating data from a large amount of data in order to identify cheating data. At present, the method of manually extracting samples is usually used to screen out cheating samples from a large amount of data, which is both time-consuming and laborious.

[0003] With the introduction of SeqGAN (Sequence Generative Adversarial Networks) that combines Generative Adversarial Networks (GAN) and reinforcement learning, SeqGAN can be used to train text generators. In the training process, the idea of ​​GAN is adopted, and a text generator is trained using existing cheating samples, and this text generator is used to generate fake cheating samples. Then, a discriminator is trained for positive and negative samples using real cheating samples and fake cheating samples respectively; then the discriminator is used to guide the generator to continuously update the network parameters so that the model generated by the generator can generate new samples that are more in line with the distribution of cheating samples; finally, the obtained generator is used to generate new cheating samples for the training of anti-cheating text models. In the process of updating network parameters, the text sample generation process is regarded as a decision-making process of a policy network (i.e., the generator network), and the policy gradient training strategy of reinforcement learning is used to solve the problem that discrete data of text type is not differentiable. The Monte Carlo sampling method is used to complete the currently generated sequence, and the probability that the completed sequence is judged as a real sample by the discriminator is used as the reward of the current Step to update the generator network, so as to gradually optimize the network parameters of the generator and generate more realistic samples.

[0004] In the above training method, the text generator uses the same hidden state parameters when generating each sample. These parameters are constantly updated during the training process, but remain unchanged when generating samples. Therefore, the same text sequence will be generated for the same initial character input. For anti-cheating scenarios, the cheating samples generated by the text generator need to have a certain degree of diversity. Therefore, the above text generator training method cannot meet the requirements of cheating sample diversity. Summary of the invention

[0005] The embodiments of the present invention provide a text generator training method and a text generator training system to solve the problem that the existing text generator training method cannot meet the requirements of cheating sample diversity.

[0006] In order to solve the above-mentioned technical problems, the present invention is achieved as follows:

[0007] In a first aspect, an embodiment of the present invention provides a text generator training method, comprising:

[0008] Perform random Gaussian initialization on the hidden layer state vector;

[0009] Reading the hidden layer state vector and the starting character as the first input of the text generator;

[0010] Generate a first character by a polynomial sampling method as a first output of the text generator;

[0011] The characters whose generation probability meets the preset conditions are used as the next output of the text generator until the end character is generated.

[0012] In a second aspect, an embodiment of the present invention provides a text generator training system, including:

[0013] Hidden layer state vector initialization module, used to perform random Gaussian initialization on the hidden layer state vector;

[0014] An input module, used for reading the hidden layer state vector and the starting character as the first input of the text generator;

[0015] A first output module, used for generating a first character as a first output of the text generator by a polynomial sampling method;

[0016] The second output module uses the characters whose generation probability meets the preset conditions as the next output of the text generator until the end character is generated.

[0017] In a third aspect, an embodiment of the present invention provides another text generator training system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor; when the processor executes the computer program, the text generator training method in the first aspect of the embodiment of the present invention is implemented.

[0018] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the text generator training method in the first aspect of the embodiment of the present invention.

[0019] In a fifth aspect, an embodiment of the present invention provides a computer program product, including a computer program, which, when executed by a processor, implements the text generator training method in the first aspect of the embodiment of the present invention.

[0020] In the embodiment of the present invention, by performing random Gaussian initialization on the hidden layer state vector, it is possible to effectively avoid the text generator from generating repeated texts, thereby improving the diversity of the generated samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flowchart of a text generator training method provided by an embodiment of the present invention;

[0022] Figure 2 1 is a schematic diagram of the training sequence of a text generator provided by an embodiment of the present invention;

[0023] Figure 3 It is a timing diagram of the pre-training stage of the text generator provided by an embodiment of the present invention;

[0024] Figure 4 is a structural schematic diagram of a text generator training system provided by an embodiment of the present invention;

[0025] Figure 5 is a structural schematic diagram of another text generator training system provided by an embodiment of the present invention;

[0026] Figure 6 It is a structural schematic diagram of another text generator training system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] like Figure 1 As shown, a text generator training method comprises the following steps:

[0029] Step 101: Perform random Gaussian initialization on the hidden layer state vector.

[0030] The hidden state vector refers to an initial value at the beginning of text generation, which is continuously updated as the next character is continuously generated. Random Gaussian initialization can also be called Gaussian distribution random initialization.

[0031] In this step, the hidden state vector is randomly initialized with Gaussian distribution, which means that when each text is generated, a random vector is calculated according to the specified Gaussian distribution as the hidden state vector, instead of directly taking an all-zero vector as the hidden state vector.

[0032] Step 102: Read in the hidden layer state vector and the starting character as the first input of the text generator.

[0033] The start character can also be called the start symbol, which refers to a special mark. As the input of the text generator, the start character can be composed of two special integer identifiers, but it does not represent any character.

[0034] The above-mentioned text generator may be composed of Long Short Term Memory (LSTM) neurons. The above-mentioned text generator may be used to generate cheating samples. Among them, cheating samples generally refer to samples containing bad information or abnormal information spread through the Internet.

[0035] Step 103: Generate the first character by a polynomial sampling method as the first output of the text generator.

[0036] The first character is generated by a polynomial sampling method, which specifically includes the following steps:

[0037] Calculating the generation probability of each character respectively through the hidden layer state vector and the starting character;

[0038] According to the generation probability of each character, each character is randomly sampled to obtain the first character.

[0039] More specifically, generating the first character (token) through the polynomial sampling method means that the text generator calculates the probabilities of all possible occurrences of the next character by reading the hidden state vector and the starting symbol (for example, the probability of "I" is 0.1%, the probability of "you" is 0.2%, and all other Chinese characters or English, numbers and other characters are similar), and then randomly samples the first character of the entire generated text according to the probability of each character (the higher the probability of a certain character, the greater the possibility of taking the character), and the first character is used as the first output of the text generator.

[0040] Step 104: taking the character whose generation probability meets the preset condition as the next output of the text generator until the end character is generated.

[0041] In this step, after the first character is generated, the characters outputted subsequently may be characters whose generation probability meets the preset conditions. Specifically, after calculating all possible probabilities of the next character, the character whose probability meets the preset conditions is taken as the next character, instead of randomly selecting one from these characters using the polynomial sampling method. When the end character is generated, it indicates that all characters have been outputted, and the set of all output characters is the sample generated by the text generator.

[0042] The generation probability satisfies a preset condition, which can be understood as the generation probability being greater than or equal to a preset threshold.

[0043] Specifically, the character whose generation probability meets the preset condition is used as the next output of the text generator, including:

[0044] The character with the highest generation probability is used as the next output of the text generator.

[0045] In this way, after the first character is generated, the subsequent output characters are directly selected from the characters with the highest generation probability. In other words, after calculating all the possible probabilities of the next character, the character with the highest probability is directly selected as the next character, instead of randomly selecting one from these characters using the polynomial sampling method.

[0046] In traditional training methods, the text generator uses the same hidden state vector when generating each sample, which is constantly updated during training but remains unchanged when generating samples. Therefore, the same text sequence will be generated for the same initial character input. For anti-cheating scenarios, the generated cheating samples need to have a certain degree of diversity. Therefore, traditional text generator training methods cannot meet the requirements of anti-cheating scenarios for sample diversity.

[0047] Figure 2A timing diagram of text generator training described in the above steps 101 to 104 is shown, wherein LSTMCell represents an LSTM neuron.

[0048] In the embodiment of the present invention, by performing random Gaussian initialization on the hidden layer state vector, it is possible to effectively avoid the text generator from generating repeated texts, thereby improving the diversity of the generated samples.

[0049] Optionally, before performing random Gaussian initialization on the hidden layer state vector, the method further includes:

[0050] The text generator is pre-trained using the first input sample to enable the text generator to achieve parameter initialization.

[0051] In order to minimize the difference between the text samples generated in the training phase and the real samples so that the discriminator will not distinguish the generated samples from the real samples too easily, the text generator needs to be pre-trained so that the parameters of the text generator are initialized reasonably.

[0052] The parameters of the text generator refer to the parameters of the deep neural network, that is, the value of the formula in each neuron that calculates the output according to the input. Different neurons have different parameter types. The parameters of all neurons in the entire deep neural network constitute the parameters of the neural network, that is, the parameters of the text generator.

[0053] The traditional method of initializing the parameters of a text generator is to optimize the policy gradient. This method guides the text generator to initialize the parameters, but it cannot generate realistic cheating samples. For tasks such as machine translation or automatic question answering, pairwise samples are generally used as input, that is, the input data of the text generator is the previous sentence of the data set sample, and the goal is to generate the next sentence of the data set sample. During the pre-training process, the parameters of the text generator are optimized by directly deriving the maximum likelihood function loss. In a single text generation task, there is no one-to-one correspondence between the two sentences in the sample data set. Therefore, the above methods cannot meet the requirements for pre-training the text generator in the embodiments of the present invention.

[0054] In view of this, embodiments of the present invention usually use random values ​​within a certain interval to initialize parameters during the training of the neural network. "For each text in the input sample data, take it as input, and use the text content obtained by right-shifting it by one character width as output" refers to training the neural network in a pre-training manner to obtain relatively suitable initialization parameters, which describes the method of constructing input-output pairs in the process of pre-training the neural network.

[0055] Among them, in order to generate reasonable initialization parameter values ​​for the text generator, the embodiment of the present invention can adopt a method similar to training a deep neural language model (Deep Neural Language Model). For each text in the first sample, it itself is used as input, and the text content obtained after the characters of each text in the first sample are moved according to preset rules is used as output, thereby achieving an effect similar to that of using pairwise text samples for training.

[0056] That is, using the first input sample to pre-train the text generator specifically includes the following steps:

[0057] Moving the characters of each text in the first sample according to a preset rule to obtain a second sample;

[0058] The second sample is used as an output of the text generator.

[0059] More specifically, the characters of each text in the first sample are moved according to a preset rule to obtain the second sample, which can be done in the following manner:

[0060] The characters of each text in the first sample are shifted right or left by N character widths to obtain a second sample, where N is a positive integer.

[0061] More specifically, the text content obtained by right-shifting the characters of each text in the first sample by one character width is output.

[0062] For example, if there is a text in the first sample that is “selling personal invoices”, the input-output pairs corresponding to shifting the characters of each text in the first sample right by one character width are as follows:

[0063] Input: Starting character Selling personal invoices

[0064] Output: End character of personal invoice for sale

[0065] Figure 3 A timing diagram of the pre-training phase of the above-mentioned text generator is shown.

[0066] By pre-training the parameters of the text generator through the above implementation method, a text sentence with a high similarity to the input sample can be generated, providing a good initial state for the subsequent optimization of the generator parameters using the supervised learning method.

[0067] It should be noted that the text generator training method of the embodiment of the present invention can not only improve the diversity of generated samples, but also make the generated samples have a high similarity with the input samples. Therefore, the text generator of the embodiment of the present invention is well suited for the generation of cheating samples. The cheating samples generated by the text generator only require a small amount of manually annotated samples, and deep learning can be used to realize the generation of cheating samples. The generated cheating samples reflect the inherent distribution of cheating, which can effectively avoid the situation where the cheating pattern mutates, thereby effectively improving the classifier's ability to identify mutated cheating samples.

[0068] like Figure 4 As shown, an embodiment of the present invention provides a text generator training system 400, including:

[0069] Hidden layer state vector initialization module 401, used for performing random Gaussian initialization on the hidden layer state vector;

[0070] An input module 402, used for reading the hidden layer state vector and the starting character as the first input of the text generator;

[0071] A first output module 403, used to generate a first character as a first output of the text generator by a polynomial sampling method;

[0072] The second output module 404 uses the characters whose generation probability meets the preset condition as the next output of the text generator until the end character is generated.

[0073] Optionally, the first output module 403 is specifically used for:

[0074] Calculating the generation probability of each character respectively through the hidden layer state vector and the starting character;

[0075] According to the generation probability of each character, each character is randomly sampled to obtain the first character.

[0076] Optionally, the second output module 404 is specifically used for:

[0077] The character with the highest generation probability is used as the next output of the text generator until the end character is generated.

[0078] Optional, such as Figure 5 As shown, the text generator training system 400 also includes:

[0079] The pre-training module 405 is used to pre-train the text generator using the input first sample so as to enable the text generator to achieve parameter initialization.

[0080] Optionally, the pre-training module 405 is specifically used for:

[0081] Moving the characters of each text in the first sample according to a preset rule to obtain a second sample;

[0082] The second sample is used as an output of the text generator.

[0083] Optionally, the pre-training module 405 is specifically used for:

[0084] The characters of each text in the first sample are shifted right or left by N character widths to obtain a second sample, where N is a positive integer.

[0085] Optionally, the text generator is composed of long short-term memory network LSTM neurons.

[0086] Optionally, the text generator is used to generate cheating samples.

[0087] It should be noted that the above-mentioned text generator training system 400 in the embodiment of the present invention can be a text generator training system of any implementation method in the method embodiment. Any implementation method in the method embodiment can be implemented by the text generator training system 400 in this embodiment and achieve the same beneficial effects. To avoid repetition, it will not be described here.

[0088] like Figure 6 As shown, another text generator training system 600 is also provided in an embodiment of the present invention. The text generator training system 600 includes a memory 601, a processor 602, and a computer program stored in the memory 601 and executable on the processor 602; when the processor 602 executes the computer program, the following steps are implemented:

[0089] Perform random Gaussian initialization on the hidden layer state vector;

[0090] Reading the hidden layer state vector and the starting character as the first input of the text generator;

[0091] Generate a first character by a polynomial sampling method as a first output of the text generator;

[0092] The characters whose generation probability meets the preset conditions are used as the next output of the text generator until the end character is generated.

[0093] exist Figure 6In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking various circuits of one or more processors represented by processor 602 and memory represented by memory 601. The bus architecture may also link various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The processor 602 is responsible for managing the bus architecture and general processing, and the memory 601 may store data used by the processor 602 when executing instructions.

[0094] Optionally, when the processor 602 executes the computer program, it further implements:

[0095] Calculating the generation probability of each character respectively through the hidden layer state vector and the starting character;

[0096] According to the generation probability of each character, each character is randomly sampled to obtain the first character.

[0097] Optionally, when the processor 602 executes the computer program, it further implements:

[0098] The character with the highest generation probability is used as the next output of the text generator until the end character is generated.

[0099] Optionally, when the processor 602 executes the computer program, it further implements:

[0100] The text generator is pre-trained using the first input sample to enable the text generator to achieve parameter initialization.

[0101] Optionally, when the processor 602 executes the computer program, it further implements:

[0102] Moving the characters of each text in the first sample according to a preset rule to obtain a second sample;

[0103] The second sample is used as an output of the text generator.

[0104] Optionally, when the processor 602 executes the computer program, it further implements:

[0105] The characters of each text in the first sample are shifted right or left by N character widths to obtain a second sample, where N is a positive integer.

[0106] Optionally, the text generator is composed of long short-term memory network LSTM neurons.

[0107] Optionally, the text generator is used to generate cheating samples.

[0108] It should be noted that the above-mentioned text generator training system 600 in the embodiment of the present invention can be a text generator training system of any implementation method in the method embodiment. Any implementation method in the method embodiment can be implemented by the text generator training system 600 in this embodiment and achieve the same beneficial effects, which will not be repeated here.

[0109] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned text generator training method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0110] An embodiment of the present invention also provides a computer program product, including a computer program, which implements the various processes of the above-mentioned text generator training method embodiment when executed by a processor.

[0111] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0112] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may be physically included separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0113] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform some steps of the sending and receiving method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0114] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A text generator training method, characterized in that: include: Perform random Gaussian initialization on the hidden layer state vector; Reading the hidden layer state vector and the starting character as the first input of the text generator; Generate a first character by a polynomial sampling method as a first output of the text generator; The characters whose generation probability meets the preset conditions are used as the next output of the text generator until the end character is generated; Before performing random Gaussian initialization on the hidden layer state vector, the method further includes: Pre-training the text generator using the first input sample so that the text generator can achieve parameter initialization; The pre-training of the text generator using the first input sample includes: Moving the characters of each text in the first sample according to a preset rule to obtain a second sample; The second sample is used as an output of the text generator.

2. The method according to claim 1, characterized in that The generating the first character by the polynomial sampling method comprises: Calculating the generation probability of each character respectively through the hidden layer state vector and the starting character; According to the generation probability of each character, each character is randomly sampled to obtain the first character.

3. The method according to claim 1 or 2, characterized in that: The generating of characters whose probability meets a preset condition as the next output of the text generator comprises: The character with the highest generation probability is used as the next output of the text generator.

4. The method according to claim 1 or 2, characterized in that: The step of moving the characters of each text in the first sample according to a preset rule to obtain a second sample includes: The characters of each text in the first sample are shifted right or left by N character widths to obtain a second sample, where N is a positive integer.

5. The method according to claim 1 or 2, characterized in that: The text generator is composed of long short-term memory network LSTM neurons.

6. The method according to claim 1 or 2, characterized in that: The text generator is used to generate cheating samples.

7. A text generator training system, characterized in that: include: Hidden layer state vector initialization module, used to perform random Gaussian initialization on the hidden layer state vector; An input module, used for reading the hidden layer state vector and the starting character as the first input of the text generator; A first output module, used for generating a first character as a first output of the text generator by a polynomial sampling method; A second output module, which uses the characters whose generation probability meets the preset conditions as the next output of the text generator until the end character is generated; A pre-training module, used for pre-training the text generator using the input first sample, so that the text generator can achieve parameter initialization; The pre-training module is specifically used for: Moving the characters of each text in the first sample according to a preset rule to obtain a second sample; The second sample is used as an output of the text generator.

8. The system according to claim 7, characterized in that The first output module is specifically used for: Calculating the generation probability of each character respectively through the hidden layer state vector and the starting character; According to the generation probability of each character, each character is randomly sampled to obtain the first character.

9. The system according to claim 7 or 8, characterized in that: The second output module is specifically used for: The character with the highest generation probability is used as the next output of the text generator until the end character is generated.

10. The system according to claim 7 or 8, characterized in that: The pre-training module is specifically used for: The characters of each text in the first sample are shifted right or left by N character widths to obtain a second sample, where N is a positive integer.

11. The system according to claim 7 or 8, characterized in that: The text generator is composed of long short-term memory network LSTM neurons.

12. The system according to claim 7 or 8, characterized in that: The text generator is used to generate cheating samples.

13. A text generator training system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor; characterized in that: When the processor executes the computer program, the text generator training method according to any one of claims 1 to 6 is implemented.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the text generator training method according to any one of claims 1 to 6 are implemented.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the text generator training method according to any one of claims 1 to 6.

Images