Model training method and device for random text generation, equipment and medium

Abstract

The invention discloses a model training method and device for random text generation, equipment and a medium, for an Auxiliary Flow module of a DFlow-T model, training text data input by the Auxiliary Flow module is all visible, and generated hidden variables are all visible as a Primary Flow module and a Decoder module respectively, so that the model is subjected to full parallel processing on input in an operation process, and the model training efficiency is improved. According to the method, the reasoning efficiency can be improved, meanwhile, the possibility of error accumulation is eradicated, the inference result has better global relevance and integrity, the DFlow is expanded to a text generation task, the advantages of the DFlow are reserved, and meanwhile the technical problem of illusion caused by error accumulation existing in an autoregression model is solved.

Description

Technical Field

[0001] The present application relates to the field of text generation technology, and in particular to a model training method, device, equipment and medium for random text generation. Background Art

[0002] In recent years, generative models have made great achievements in various fields. Existing generative frameworks mainly include: Generative Adversarial Network (GAN) and likelihood-based models, such as auto-regressive models (AR), normalizing flow models (NF), diffusion models, etc.

[0003] In the field of text generation, the most popular one is the GPT model, which is essentially an autoregressive model. However, due to its structural characteristics, the autoregressive model faces a difficult problem to solve: "hallucination" caused by error accumulation. During the reasoning process, the output of the autoregressive model at the current moment depends on all the previous input content. So, if the output of the current step is slightly biased, then the next step will be generated based on the biased (wrong) content. It is not difficult to understand that the output result is more likely to be wrong at this time. In this way, the problem of error accumulation arises. In text generation, this can easily lead to the text content output by the model describing information that does not match the facts, which is the so-called "hallucination" problem. For the autoregressive model, this problem is determined by the structural characteristics and is essentially difficult to solve. In addition, the autoregressive model also has the problem of slow inference speed.

[0004] Diffusion models have also achieved good results in the field of text generation recently. The non-autoregressive structure improves the "hallucination problem" and significantly improves the generation effect. However, its diffusion process also leads to the problem of slow inference speed.

[0005] In previous work, a new generative framework was proposed: Denoising Flow (DFlow). This structure cleverly combines the flow model (Flow) and the Denoising Autoencoder (DAE), and has achieved excellent results in audio generation tasks. Compared with other generative frameworks, DFlow has many advantages:

[0006] 1. Efficient training and inference: Compared with autoregressive models or diffusion models, DFlow's training and inference are all parallel structures with efficient operations. At the same time, DFlow's training and inference have low computational complexity, allowing it to use a larger batch size during training, which not only makes training more stable, but also promotes faster model convergence. In addition, the low computational complexity allows DFlow to occupy less memory resources during inference and be more efficient.

[0007] 2. Due to the combination of DAE, the generation effect of DFlow is not only significantly optimized compared to the traditional Flow model, but also has a significant improvement in the robustness of the model.

[0008] 3. DFlow inherits the characteristics of traditional Flow models with explicit and controllable hidden variables, making it easier to connect to various downstream tasks.

[0009] 4. Simple structure and stable training: Compared with the GAN series models that are prone to mode collapse, the DFlow model is not only simple in structure and highly interpretable, but also very stable in training, making it easy to expand into a large model, making DFlow a very promising generative framework.

[0010] Based on DFlow, how to expand DFlow to text generation tasks to solve the "hallucination" caused by the accumulation of errors in the autoregressive model while retaining the advantages of DFlow is a technical problem that technicians in this field urgently need to solve. Summary of the invention

[0011] The present application provides a model training method, apparatus, device and medium for random text generation, which realizes the expansion of DFlow to text generation tasks, retains the advantages of DFlow, and solves the technical problem of "hallucination" caused by error accumulation in autoregressive models.

[0012] In view of this, the first aspect of the present application provides a model training method for random text generation, the method comprising:

[0013] Step S1, obtaining training text data x;

[0014] Step S2: Take the training text data x as input and input it into the Auxiliary Flow module of the DFlow-T model to obtain the latent variable The Auxiliary Flow module adopts the causal-based Transformer structure as the basic structure for implementing Flow;

[0015] Step S3: The Primary Flow module of the DFlow-T model performs the following operations on the latent variable z:l Perform forward calculation to get z p =g(z l ), and z p The distribution of The Primary Flow module uses the Causal Transformer structure as the basic structure for implementing Flow;

[0016] Step S4: The Decoder module of the DFlow-T model is based on the latent variable z l Reconstruct the training text data x to obtain training reconstructed text data The Decoder module uses a multi-layer CausalTransformer structure as the basic structure for implementing Flow;

[0017] Step S5: construct the loss function of the DFlow-T model, wherein the loss function includes an input loss function based on maximizing likelihood estimation and a loss function of the training text data x and the training reconstruction text data obtained based on cross entropy calculation. The reconstruction loss function between

[0018] Step S6: Based on the loss function, update the parameters of the AuxiliaryFlow module, the Primary Flow module and the Decoder module of the DFlow-T model until the loss function converges to obtain the trained DFlow-T model.

[0019] Optionally, the AuxiliaryFlow module and the PrimaryFlow module are reversible flow networks;

[0020] The AuxiliaryFlow module and the PrimaryFlow module constitute a Flow structure.

[0021] Optionally, the AuxiliaryFlow module and the Decoder module constitute an AE structure.

[0022] Optionally, the input loss function based on maximizing likelihood estimation is specifically:

[0023] log p(x)=log(p(z p )+log det|J(f)|+log det|J(g)|

[0024] Where p(x) is the probability density function, which indicates the probability of x occurring, z p The distribution of det|J(f)| and det|J(g)| represent the Jacobian matrix determinants of f and g, respectively.

[0025] Optionally, the training text data x obtained by cross entropy calculation is consistent with the training reconstruction text data The reconstruction loss function between is specifically:

[0026] Optionally, the step S1 specifically includes:

[0027] The training text content is obtained, and after preprocessing the training text content, the training text content is converted into vector-type training text data x.

[0028] The second aspect of the present application provides a random text generation method, which uses the DFlow-T model trained by the model training method for random text generation provided in the first aspect of the present application, and the method includes:

[0029] Randomly sample a point z from a standard Gaussian distribution p As input, it is input into the PrimaryFlow module of the DFlow-T model;

[0030] The PrimaryFlow module of the DFlow-T model is for the point z p Perform the inverse operation to obtain the hidden variable z l =g -1 (z p );

[0031] The Decoder module of the DFlow-T model performs the operation on the latent variable z l Generate text and generate random text data

[0032] A third aspect of the present application provides a model training device for random text generation, the device comprising:

[0033] An acquisition unit, used for acquiring training text data x;

[0034] The first processing unit is used to input the training text data x as input into the Auxiliary Flow module of the DFlow-T model to obtain the hidden variable The Auxiliary Flow module adopts the causal-based Transformer structure as the basic structure for implementing Flow;

[0035] The second processing unit is used for the PrimaryFlow module of the DFlow-T model to process the latent variable zl Perform forward calculation to get z p =g(z l ), and z p The distribution of The Primary Flow module uses the Causal Transformer structure as the basic structure for implementing Flow;

[0036] The third processing unit is used for the Decoder module of the DFlow-T model according to the latent variable z l Reconstruct the training text data x to obtain training reconstructed text data The Decoder module uses a multi-layer CausalTransformer structure as the basic structure for implementing Flow;

[0037] A loss function construction unit is used to construct the loss function of the DFlow-T model, wherein the loss function includes an input loss function based on maximizing likelihood estimation and a loss function of the training text data x and the training reconstruction text data obtained based on cross entropy calculation. The reconstruction loss function between

[0038] A parameter updating unit is used to update the parameters of the AuxiliaryFlow module, the Primary Flow module and the Decoder module of the DFlow-T model based on the loss function until the loss function converges to obtain the trained DFlow-T model.

[0039] Optionally, the Auxiliary Flow module and the Primary Flow module are reversible flow networks;

[0040] The AuxiliaryFlow module and the PrimaryFlow module constitute a Flow structure.

[0041] Optionally, the Auxiliary Flow module and the Decoder module constitute an AE structure.

[0042] Optionally, the input loss function based on maximizing likelihood estimation is specifically:

[0043] log p(x)=log(p(z p )+log det|J(f)|+log det|J(g)|

[0044] Where p(x) is the probability density function, which indicates the probability of x occurring, z pThe distribution of det|J(f)| and det|J(g)| represent the Jacobian matrix determinants of f and g, respectively.

[0045] A fourth aspect of the present application provides a model training device for random text generation, the device comprising a processor and a memory:

[0046] The memory is used to store program code and transmit the program code to the processor;

[0047] The processor is used to execute the steps of the method for model training for random text generation as described in the first aspect above according to the instructions in the program code.

[0048] A fifth aspect of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store program code, and the program code is used to execute the method described in the first aspect above.

[0049] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0050] In the present application, a model training method, device, equipment and medium for random text generation are provided. For the Auxiliary Flow module of the DFlow-T model, the input training text data is fully visible, and the generated hidden variables are fully visible as the Primary Flow module and the Decoder module, respectively, so that the model processes the input in full parallel during the operation, which can not only improve the reasoning efficiency, but also eliminate the possibility of error accumulation. The inference results have better global correlation and integrity, and the DFlow is extended to the text generation task, retaining the advantages of DFlow, while solving the technical problem of "hallucination" caused by error accumulation in the autoregressive model. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is the network architecture diagram of the DFlow-T model;

[0052] Figure 2 A method flow chart of a model training method for random text generation in an embodiment of the present application;

[0053] Figure 3 This is a schematic diagram of the structure of a model training device for random text generation in an embodiment of the present application;

[0054] Figure 4 This is a schematic diagram of the structure of a model training device for random text generation in an embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0056] This application designs a model training method, device, equipment and medium for random text generation, which realizes the expansion of DFlow to text generation tasks, retains the advantages of DFlow, and solves the technical problem of "hallucination" caused by error accumulation in autoregressive models.

[0057] This application provides a DFlow-T model, such as Figure 1 As shown, Figure 1 The network architecture diagram of the DFlow-T model, where the solid line is the training process of the model and the dotted line is the reasoning process of the model. The DFlow-T model includes:

[0058] Auxiliary Flow module 101, Primary Flow module 102 and Decoder module 103;

[0059] Among them, the AuxiliaryFlow module 101 adopts the causal-based Transformer structure as the basic structure for implementing Flow, the Primary Flow module 102 adopts the CausalTransformer structure as the basic structure for implementing Flow, and the Decoder module 103 adopts a multi-layer Causal Transformer structure as the basic structure for implementing Flow.

[0060] It should be noted that in order to extend the powerful structure of DFlow to the field of NLP, it is applied to the task of text segment generation. Figure 1 As shown in the figure, the overall structure of the DFlow-T model maintains the original structure of three modules: AuxiliaryFlow f, PrimaryFlow g, and Decoder m.

[0061] For each module, we have optimized and improved the specific structure. The following are the details and design principles of the improvement:

[0062] AuxiliaryFlow module 101, whose operation process is represented by f: This module is a non-volume preserving flow. Compared with volume preserving flow, volume preserving flow has stronger expression ability and requires fewer network layers. In order to adapt to the text generation task, AuxiliaryFlow module 101 uses a causal transformer structure as the basic structure for implementing Flow. The core of the transformer structure lies in its self-attention mechanism, which has a very strong content capture ability. Based on the causal transformer structure, when calculating self-attention for a text sequence, the current text is only related to the previous text and has nothing to do with the subsequent text. This design is in line with the logic of human language. At the same time, for the auxiliary flow module 101, during the training process, the input text data is all visible. Therefore, when operating on each character, the causal transformer structure will generate hidden variables based on all known inputs, avoiding the error accumulation that is easy to occur in the word-by-word reasoning process of the autoregressive model.

[0063] Primary Flow module 102, whose operation process is represented by g: This module is a volume invariant flow, the purpose of which is to maintain the implicit variable z l The regularization effect is achieved to ensure the training stability of the model. However, its internal infrastructure is also changed to the Causal Transformer structure, which ensures the model's content learning and content generation capabilities.

[0064] The operation process of the decoder module 103 is represented by m: In the text generation task, text features are all discrete features. Therefore, unlike image generation or audio generation, text generation does not need to consider the problem of "detail generation". Therefore, in the decoder module, the model is given the ability to generate content by using a multi-layer CausalTransformer structure.

[0065] For easier understanding, see Figure 2 , Figure 2 This is a method flow chart of the model training method for random text generation in an embodiment of the present application, such as Figure 2 As shown, specifically:

[0066] Step S1, obtaining training text data x;

[0067] It should be noted that the training text data x is usually pre-processed and is not directly composed of a piece of text, but the text is converted into vector input data. Specifically:

[0068] Collect text data for training models;

[0069] Clean the collected text, including removing noise, standardizing text, etc.;

[0070] Convert text into numerical data that the model can process, which may include word embeddings or sentence embeddings.

[0071] Step S2: Take the training text data x as input and input it into the Auxiliary Flow module of the DFlow-T model to obtain the hidden variable z l =f(x), the Auxiliary Flow module uses the causal-based Transformer structure as the basic structure for implementing Flow;

[0072] It should be noted that the purpose of the Auxiliary Flow module is to convert the input vector data into latent variables. Its operation process is related to its network structure and parameters. During the training process, the corresponding parameters will be optimized according to the loss function. f() is specifically the neural network calculation process of Auxiliary Flow.

[0073] Step S3: The Primary Flow module of the DFlow-T model performs the following operations on the latent variable z: l Perform forward calculation to get z p =g(z l ), and z p The distribution of The Primary Flow module uses the CausalTransformer structure as the basic structure for implementing Flow;

[0074] It should be noted that during the training process, the hidden variables output by the AuxiliaryFlow module are forward calculated through the Primary Flow module to obtain z p =g(z l ), the purpose of training is to make z p The distribution of So that in the inference stage, a point randomly sampled in accordance with the standard Gaussian distribution can be used as the input for random text generation. g() is specifically the neural network calculation process of Primary Flow.

[0075] Step S4: The Decoder module of the DFlow-T model calculates the hidden variable z l Reconstruct the training text data x to obtain the training reconstructed text data The Decoder module uses a multi-layer Causal Transformer structure as the basic structure to implement Flow;

[0076] It should be noted that during the training process, the Decoder module reconstructs the text based on the latent variables output by the AuxiliaryFlow module. Its goal is to make the reconstructed text data closer and closer to the original input training text data. The best result is that it is consistent with the original input training text data. m() is specifically the neural network calculation process of the Decoder.

[0077] Step S5: construct the loss function of the DFlow-T model. The loss function includes the input loss function based on maximizing the likelihood estimation and the training text data x and the training reconstruction text data calculated based on cross entropy. The reconstruction loss function between

[0078] Step S6: Update the parameters of the Auxiliary Flow module, Primary Flow module, and Decoder module of the DFlow-T model based on the loss function until the loss function converges to obtain the trained DFlow-T model.

[0079] It should be noted that the parameters of the model are iteratively updated based on the loss function. When the loss function reaches a lower value or stabilizes, the update stops, and the trained and parameter-optimized DFlow-T model is obtained.

[0080] Furthermore, the Auxiliary Flow module and the Primary Flow module are reversible flow networks;

[0081] The Auxiliary Flow module and the Primary Flow module constitute the Flow structure.

[0082] Furthermore, the Auxiliary Flow module and the Decoder module constitute the AE structure.

[0083] It should be noted that the AE structure consists of AuxiliaryFlow and Decoder.

[0084] It is worth mentioning that, unlike the original DFLow, in order to adapt to the text generation task, this paper replaces the original DAE structure with the AE structure. The AE structure refers to the abbreviation of auto-encoder, which is translated as "self-encoder".

[0085] In image or audio generation tasks, the input and output of the model are continuous signals, so the model needs to take into account both the ability to generate content and details. The denoising autoencoder can effectively guarantee the ability to generate details while also improving the robustness of the model.

[0086] However, in the text generation task, the model input is a discrete signal, and the generated signal is also a discrete signal. Therefore, the model needs to focus on content generation without considering the problem of detail generation. In this case, we do not need to perform "noise processing" on the model input similar to the original text.

[0087] On the other hand, some text generation frameworks (such as BERT) give the model the ability to learn content by adding masks (adding masks can also be understood as a way of adding noise), but these two types of models do not have the ability to generate instructions, that is, they do not have the ability to perform subsequent generation based on prompts, which makes the scope of use of the model very limited. The ability of autoregressive models similar to GPT to generate based on instructions enables the model to be widely used. The essence of the autoregressive model is to use the causal network structure. During training, since the inputs of each time step are known (the input sequence is all visible), parallel operations can be performed. However, during inference, the operation of the next time step depends on the output of the previous time step, so it must be performed in an autoregressive manner. In addition, the causal structure can also be understood as a noise addition scheme, that is, masking the content after the current time step.

[0088] Furthermore, the input loss function based on maximizing likelihood estimation is specifically:

[0089] log p(x)=log(p(z p )+log det|J(f)|+log det|J(g)|

[0090] Where p(x) is the probability density function, which indicates the probability of x occurring, z p The distribution of det|J(f)| and det|J(g)| represent the Jacobian matrix determinants of f and g, respectively.

[0091] It should be noted that the log-likelihood of the input x during training can be calculated using the following formula:

[0092] log p(x)=log(p(z p )+log det|J(f)|+log det|J(g)|

[0093] in det|J(f)| and det|J(g)| represent the Jacobian matrix determinants of f and g, respectively.

[0094] During training, we use the maximum likelihood estimation as the loss function for this part. mle .

[0095] The DFlow-T model of this application uses the causal + self-attention network structure, which gives the model powerful content learning and generation capabilities, as well as the ability to generate based on instructions. In addition, thanks to the ingenious structural design of DFlow-T, the entire network is fully parallel in the training and inference stages. During training, x is fully visible, so z l The calculation is completely parallel, so The calculation of is also fully parallel. At the same time, we also trained another network PrimaryFlow for parallel reasoning z l Specifically, during inference we first randomly sample a z from a standard Gaussian distribution p , when using PrimaryFlow to calculate z l Time p is fully visible, so z l The calculations are parallel, and thus the entire reasoning process is parallel. This not only solves the problem of slow reasoning of the GPT model or diffusion model, but also solves the fatal weakness of the GPT model that errors are easily accumulated and lead to "hallucinations".

[0096] The present application also provides a random text generation method, which uses the DFlow-T model trained by the model training method for random text generation provided in the present application, and the method includes:

[0097] Randomly sample a point z from a standard Gaussian distribution p As input, it is fed into the PrimaryFlow module of the DFlow-T model;

[0098] The Primary Flow module of the DFlow-T model is for point z p Perform the inverse operation to obtain the hidden variable z l =g -1 (z p );

[0099] The Decoder module of the DFlow-T model performs the calculation on the latent variable z l Generate text and generate random text data

[0100] It should be noted that when random text generation is required, the DFlow-T model trained in the embodiment of the present application can randomly sample a point z from a standard Gaussian distribution. p As the input of the Primary Flow module, the hidden variable z can be obtained by inverse operation l , and the Decoder module is based on the input hidden variable z l Reconstruct the text to generate the corresponding random sampling point z p Random text data

[0101] As a text generation method, random sampling can improve the variability of text generation.

[0102] See also Figure 3 , Figure 3 Schematic diagram of the structure of the model training device for random text generation in the embodiment of the present application, such as Figure 3 As shown, specifically:

[0103] An acquisition unit 301 is used to acquire training text data x;

[0104] It should be noted that the training text data x is usually pre-processed and is not directly composed of a piece of text, but the text is converted into vector input data. Specifically:

[0105] Collect text data for training models;

[0106] Clean the collected text, including removing noise, standardizing text, etc.;

[0107] Convert text into numerical data that the model can process, which may include word embeddings or sentence embeddings.

[0108] The first processing unit 302 is used to input the training text data x as input to the AuxiliaryFlow module of the DFlow-T model to obtain the hidden variable z l =f(x), the AuxiliaryFlow module uses the causal-based Transformer structure as the basic structure for implementing Flow;

[0109] It should be noted that the purpose of the Auxiliary Flow module is to convert the input vector data into latent variables. Its operation process is related to its network structure and parameters. The corresponding parameters during training will be optimized according to the loss function.

[0110] The second processing unit 303 is used for the Primary Flow module of the DFlow-T model to process the latent variable z lPerform forward calculation to get z p =g(z l ), and z p The distribution of The PrimaryFlow module uses the CausalTransformer structure as the basic structure for implementing Flow;

[0111] It should be noted that during the training process, the hidden variables output by the AuxiliaryFlow module are forward calculated through the Primary Flow module to obtain z p =g(z l ), the purpose of training is to make z p The distribution of So that in the inference stage, a point randomly sampled from a standard Gaussian distribution can be used as the input for random text generation.

[0112] The third processing unit 304, a Decoder module for the DFlow-T model, calculates the hidden variable z l Reconstruct the training text data x to obtain the training reconstructed text data The Decoder module uses a multi-layer CausalTransformer structure as the basic structure for implementing Flow;

[0113] It should be noted that during the training process, the Decoder module reconstructs the text based on the latent variables output by the AuxiliaryFlow module. Its goal is to make the reconstructed text data closer and closer to the original input training text data, and it is best if it is consistent with the original input training text data.

[0114] The loss function construction unit 305 is used to construct the loss function of the DFlow-T model. The loss function includes an input loss function based on maximizing likelihood estimation and a loss function of the training text data x and the training reconstruction text data obtained by cross entropy calculation. The reconstruction loss function between

[0115] The parameter updating unit 306 is used to update the parameters of the AuxiliaryFlow module, the PrimaryFlow module and the Decoder module of the DFlow-T model based on the loss function until the loss function converges to obtain the trained DFlow-T model.

[0116] It should be noted that the parameters of the model are iteratively updated based on the loss function. When the loss function reaches a lower value or stabilizes, the update stops, and the trained and parameter-optimized DFlow-T model is obtained.

[0117] Furthermore, the Auxiliary Flow module and the Primary Flow module are reversible flow networks;

[0118] The AuxiliaryFlow module and the PrimaryFlow module constitute the Flow structure.

[0119] Furthermore, the AuxiliaryFlow module and the Decoder module constitute the AE structure.

[0120] It should be noted that the AE structure consists of AuxiliaryFlow and Decoder.

[0121] It is worth mentioning that, unlike the original DFLow, in order to adapt to the text generation task, this paper replaces the original DAE structure with the AE structure.

[0122] In image or audio generation tasks, the input and output of the model are continuous signals, so the model needs to take into account both the ability to generate content and details. The denoising autoencoder can effectively guarantee the ability to generate details while also improving the robustness of the model.

[0123] However, in the text generation task, the model input is a discrete signal, and the generated signal is also a discrete signal. Therefore, the model needs to focus on content generation without considering the problem of detail generation. In this case, we do not need to perform "noise processing" on the model input similar to the original text.

[0124] On the other hand, some text generation frameworks (such as BERT) give the model the ability to learn content by adding masks (adding masks can also be understood as a way of adding noise), but these two types of models do not have the ability to generate instructions, that is, they do not have the ability to perform subsequent generation based on prompts, which makes the scope of use of the model very limited. The ability of autoregressive models similar to GPT to generate based on instructions enables the model to be widely used. The essence of the autoregressive model is to use the causal network structure. During training, since the inputs of each time step are known (the input sequence is all visible), parallel operations can be performed. However, during inference, the operation of the next time step depends on the output of the previous time step, so it must be performed in an autoregressive manner. In addition, the causal structure can also be understood as a noise addition scheme, that is, masking the content after the current time step.

[0125] Furthermore, the input loss function based on maximizing likelihood estimation is specifically:

[0126] log p(x)=log(p(z p)+log det|J(f)|+log det|J(g)|

[0127] Where p(x) is the probability density function, which indicates the probability of x occurring, z p The distribution of det|J(f)| and det|J(g)| represent the Jacobian matrix determinants of f and g, respectively.

[0128] It should be noted that the log-likelihood of the input x during training can be calculated using the following formula:

[0129] log p(x)=log(p(z p )+log det|J(f)|+log det|J(g)|

[0130] Where p(x) is the probability density function, which indicates the probability of x occurring, z p The distribution of det|J(f)| and det|J(g)| represent the Jacobian matrix determinants of f and g, respectively.

[0131] During training, we use the maximum likelihood estimation as the loss function for this part. mle .

[0132] The DFlow-T model of this application uses the causal + self-attention network structure, which gives the model powerful content learning and generation capabilities, as well as the ability to generate based on instructions. In addition, thanks to the ingenious structural design of DFlow-T, the entire network is fully parallel in the training and inference stages. During training, x is fully visible, so z l The calculation is completely parallel, so The calculation of is also fully parallel. At the same time, we also trained another network PrimaryFlow for parallel reasoning z l Specifically, during inference we first randomly sample a z from a standard Gaussian distribution p , when using PrimaryFlow to calculate z l Time p is fully visible, so z l The calculations are parallel, and thus the entire reasoning process is parallel. This not only solves the problem of slow reasoning of the GPT model or diffusion model, but also solves the fatal weakness of the GPT model that errors are easily accumulated and lead to "hallucinations".

[0133] The present application also provides another model training device for random text generation, such as Figure 4 For the sake of convenience, only the parts related to the embodiments of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiments of the present application. The terminal can be any terminal device including a mobile phone, a tablet computer, a personal digital assistant (English full name: Personal Digital Assistant, English abbreviation: PDA), a sales terminal (English full name: Point of Sales, English abbreviation: POS), a car computer, etc., taking the terminal as a mobile phone as an example:

[0134] Figure 4 FIG. 1 is a block diagram showing a partial structure of a mobile phone related to a terminal provided in an embodiment of the present application. Figure 4 The mobile phone includes: a radio frequency (full name: Radio Frequency, English abbreviation: RF) circuit 1010, a memory 1020, an input unit 1030, a display unit 1040, a sensor 1050, an audio circuit 1060, a wireless fidelity (full name: wireless fidelity, English abbreviation: WiFi) module 1070, a processor 1080, and a power supply 1090. Those skilled in the art can understand that Figure 4 The mobile phone structure shown in the figure does not constitute a limitation on the mobile phone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0135] Combine the following Figure 4 A detailed introduction to the various components of the mobile phone:

[0136] The RF circuit 1010 can be used for receiving and sending signals during information transmission or calls. In particular, after receiving the downlink information of the base station, it is sent to the processor 1080 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit 1010 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (full name: Low Noise Amplifier, English abbreviation: LNA), a duplexer, etc. In addition, the RF circuit 1010 can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communications can use any communication standard or protocol, including but not limited to Global System of Mobile communications (Global System of Mobile communication, English abbreviation: GSM), General Packet Radio Service (General Packet Radio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, English abbreviation: CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (Long Term Evolution, English abbreviation: LTE), e-mail, Short Messaging Service (SMS), etc.

[0137] The memory 1020 can be used to store software programs and modules. The processor 1080 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 1020. The memory 1020 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 1020 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0138] The input unit 1030 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile phone. Specifically, the input unit 1030 may include a touch panel 1031 and other input devices 1032. The touch panel 1031, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 1031 or near the touch panel 1031 using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel 1031 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the processor 1080, and can receive and execute commands sent by the processor 1080. In addition, the touch panel 1031 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 1031, the input unit 1030 may further include other input devices 1032. Specifically, the other input devices 1032 may include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

[0139] The display unit 1040 can be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 1040 may include a display panel 1041. Optionally, the display panel 1041 may be configured in the form of a liquid crystal display (full name in English: Liquid Crystal Display, English abbreviation: LCD), an organic light-emitting diode (full name in English: Organic Light-Emitting Diode, English abbreviation: OLED), etc. Further, the touch panel 1031 may cover the display panel 1041. When the touch panel 1031 detects a touch operation on or near it, it is transmitted to the processor 1080 to determine the type of touch event. Subsequently, the processor 1080 provides a corresponding visual output on the display panel 1041 according to the type of touch event. Although in Figure 4 In the embodiment, the touch panel 1031 and the display panel 1041 are used as two independent components to realize the input and output functions of the mobile phone, but in some embodiments, the touch panel 1031 and the display panel 1041 can be integrated to realize the input and output functions of the mobile phone.

[0140] The mobile phone may also include at least one sensor 1050, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 1041 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 1041 and / or the backlight when the mobile phone is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0141] The audio circuit 1060, the speaker 1061, and the microphone 1062 can provide an audio interface between the user and the mobile phone. The audio circuit 1060 can transmit the received audio data to the speaker 1061 after converting the received audio data into an electrical signal, which is converted into a sound signal for output; on the other hand, the microphone 1062 converts the collected sound signal into an electrical signal, which is received by the audio circuit 1060 and converted into audio data, and then the audio data is output to the processor 1080 for processing, and then sent to another mobile phone through the RF circuit 1010, or the audio data is output to the memory 1020 for further processing.

[0142] WiFi is a short-range wireless transmission technology. The mobile phone can help users send and receive emails, browse web pages and access streaming media through the WiFi module 1070. It provides users with wireless broadband Internet access. Figure 4 A WiFi module 1070 is shown, but it is understandable that it is not an essential component of the mobile phone and can be omitted as needed without changing the essence of the invention.

[0143] The processor 1080 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. By running or executing software programs and / or modules stored in the memory 1020 and calling data stored in the memory 1020, it executes various functions of the mobile phone and processes data, thereby monitoring the mobile phone as a whole. Optionally, the processor 1080 may include one or more processing units; preferably, the processor 1080 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1080.

[0144] The mobile phone also includes a power supply 1090 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 1080 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption.

[0145] Although not shown, the mobile phone may also include a camera, a Bluetooth module, etc., which will not be described in detail here.

[0146] In the embodiment of the present application, the processor 1080 included in the terminal also has the following functions:

[0147] Step S1, obtaining training text data x;

[0148] It should be noted that the training text data x is usually pre-processed and is not directly composed of a piece of text, but the text is converted into vector input data. Specifically:

[0149] Collect text data for training models;

[0150] Clean the collected text, including removing noise, standardizing text, etc.;

[0151] Convert text into numerical data that the model can process, which may include word embeddings or sentence embeddings.

[0152] Step S2: Take the training text data x as input and input it into the Auxiliary Flow module of the DFlow-T model to obtain the hidden variable z l =f(x), the Auxiliary Flow module uses the causal-based Transformer structure as the basic structure for implementing Flow;

[0153] It should be noted that the purpose of the Auxiliary Flow module is to convert the input vector data into latent variables. Its operation process is related to its network structure and parameters. The corresponding parameters during training will be optimized according to the loss function.

[0154] Step S3: The Primary Flow module of the DFlow-T model performs the following operations on the latent variable z: l Perform forward calculation to get z p =g(z l ), and z p The distribution of The Primary Flow module uses the CausalTransformer structure as the basic structure for implementing Flow;

[0155] It should be noted that during the training process, the hidden variables output by the AuxiliaryFlow module are forward calculated through the Primary Flow module to obtain z p =g(z l ), the purpose of training is to make z p The distribution of So that in the inference stage, a point randomly sampled from a standard Gaussian distribution can be used as the input for random text generation.

[0156] Step S4: The Decoder module of the DFlow-T model calculates the hidden variable z l Reconstruct the training text data x to obtain the training reconstructed text data The Decoder module uses a multi-layer Causal Transformer structure as the basic structure to implement Flow;

[0157] It should be noted that during the training process, the Decoder module reconstructs the text based on the latent variables output by the AuxiliaryFlow module. Its goal is to make the reconstructed text data closer and closer to the original input training text data, and it is best if it is consistent with the original input training text data.

[0158] Step S5: construct the loss function of the DFlow-T model. The loss function includes the input loss function based on maximizing the likelihood estimation and the training text data x and the training reconstruction text data calculated based on cross entropy. The reconstruction loss function between

[0159] Step S6: Update the parameters of the Auxiliary Flow module, Primary Flow module, and Decoder module of the DFlow-T model based on the loss function until the loss function converges to obtain the trained DFlow-T model.

[0160] It should be noted that the parameters of the model are iteratively updated based on the loss function. When the loss function reaches a lower value or stabilizes, the update stops, and the trained and parameter-optimized DFlow-T model is obtained.

[0161] Furthermore, the Auxiliary Flow module and the Primary Flow module are reversible flow networks;

[0162] The Auxiliary Flow module and the Primary Flow module constitute the Flow structure.

[0163] Furthermore, the Auxiliary Flow module and the Decoder module constitute the AE structure.

[0164] It should be noted that the AE structure consists of AuxiliaryFlow and Decoder.

[0165] It is worth mentioning that, unlike the original DFLow, in order to adapt to the text generation task, this paper replaces the original DAE structure with the AE structure.

[0166] In image or audio generation tasks, the input and output of the model are continuous signals, so the model needs to take into account both the ability to generate content and details. The denoising autoencoder can effectively guarantee the ability to generate details while also improving the robustness of the model.

[0167] However, in the text generation task, the model input is a discrete signal, and the generated signal is also a discrete signal. Therefore, the model needs to focus on content generation without considering the problem of detail generation. In this case, we do not need to perform "noise processing" on the model input similar to the original text.

[0168] On the other hand, some text generation frameworks (such as BERT) give the model the ability to learn content by adding masks (adding masks can also be understood as a way of adding noise), but these two types of models do not have the ability to generate instructions, that is, they do not have the ability to perform subsequent generation based on prompts, which makes the scope of use of the model very limited. The ability of autoregressive models similar to GPT to generate based on instructions enables the model to be widely used. The essence of the autoregressive model is to use the causal network structure. During training, since the inputs of each time step are known (the input sequence is all visible), parallel operations can be performed. However, during inference, the operation of the next time step depends on the output of the previous time step, so it must be performed in an autoregressive manner. In addition, the causal structure can also be understood as a noise addition scheme, that is, masking the content after the current time step.

[0169] Furthermore, the input loss function based on maximizing likelihood estimation is specifically:

[0170] log p(x)=log(p(z p )+log det|J(f)|+log det|J(g)|

[0171] Where p(x) is the probability density function, which indicates the probability of x occurring, z p The distribution of det|J(f)| and det|J(g)| represent the Jacobian matrix determinants of f and g, respectively.

[0172] An embodiment of the present application also provides a computer-readable storage medium for storing program code, which is used to execute any implementation of a model training method for random text generation described in the aforementioned embodiments.

[0173] In the embodiments of the present application, a model training method, apparatus, device and medium for random text generation are provided. For the AuxiliaryFlow module of the DFlow-T model, the input training text data is fully visible, and the generated hidden variables are fully visible as the Primary Flow module and the Decoder module, respectively, so that the model processes the input in full parallel during the operation, which can not only improve the reasoning efficiency, but also eliminate the possibility of error accumulation. The inference results have better global correlation and integrity, and realize the expansion of DFlow to text generation tasks, retaining the advantages of DFlow, and solving the technical problem of "hallucination" caused by error accumulation in the autoregressive model.

[0174] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0175] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0176] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0177] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0178] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0179] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0180] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (full name in English: Read-Only Memory, English abbreviation: ROM), random access memory (full name in English: Random Access Memory, English abbreviation: RAM), disk or optical disk and other media that can store program codes.

[0181] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A model training method for random text generation, characterized in that: include: Step S1, obtaining training text data x; Step S2: Take the training text data x as input and input it into the Auxiliary Flow module of the DFlow-T model to obtain the hidden variable z l =f(x), the Auxiliary Flow module adopts the causal-based Transformer structure as the basic structure for implementing Flow; Step S3: The Primary Flow module of the DFlow-T model performs the following operations on the latent variable z: l Perform forward calculation to get z p =g(z l ), and z p The distribution of The Primary Flow module uses the CausalTransformer structure as the basic structure for implementing Flow; Step S4: The Decoder module of the DFlow-T model is based on the latent variable z l Reconstruct the training text data x to obtain training reconstructed text data The Decoder module uses a multi-layer CausalTransformer structure as the basic structure for implementing Flow; Step S5: construct the loss function of the DFlow-T model, wherein the loss function includes an input loss function based on maximizing likelihood estimation and a loss function of the training text data x and the training reconstruction text data obtained based on cross entropy calculation. The reconstruction loss function between Step S6: Based on the loss function, update the parameters of the AuxiliaryFlow module, the Primary Flow module and the Decoder module of the DFlow-T model until the loss function converges to obtain the trained DFlow-T model.

2. The model training method for random text generation according to claim 1, characterized in that: The AuxiliaryFlow module and the PrimaryFlow module are reversible flow networks; The AuxiliaryFlow module and the PrimaryFlow module constitute a Flow structure.

3. The model training method for random text generation according to claim 2, characterized in that: The AuxiliaryFlow module and the Decoder module constitute the AE structure.

4. The model training method for random text generation according to claim 1, characterized in that: The input loss function based on maximizing likelihood estimation is specifically: logp(x)=log p(z p )+log it|J(f)|+log it|J(g)| Where p(x) is the probability density function, which indicates the probability of x occurring, z p The distribution of det|J(f)| and det|J(g)| represent the Jacobian matrix determinants of f and g, respectively.

5. The model training method for random text generation according to claim 1, characterized in that: The training text data x obtained by cross entropy calculation and the training reconstructed text data The reconstruction loss function between is specifically:

6. The model training method for random text generation according to claim 1, characterized in that: The step S1 specifically includes: The training text content is obtained, and after preprocessing the training text content, the training text content is converted into vector-type training text data x.

7. A random text generation method, characterized in that: The DFlow-T model trained by the model training method for random text generation according to any one of claims 1 to 6, wherein the random text generation method comprises: Randomly sample a point z from a standard Gaussian distribution p As input, it is input into the PrimaryFlow module of the DFlow-T model; The PrimaryFlow module of the DFlow-T model is for the point z p Perform the inverse operation to obtain the hidden variable z l =g -1 (z p ); The Decoder module of the DFlow-T model performs the operation on the latent variable z l Generate text and generate random text data 8. A model training device for random text generation, characterized in that: include: An acquisition unit, used for acquiring training text data x; The first processing unit is used to input the training text data x as input into the AuxiliaryFlow module of the DFlow-T model to obtain the hidden variable The Auxiliary Flow module adopts the causal-based Transformer structure as the basic structure for implementing Flow; The second processing unit is used for the PrimaryFlow module of the DFlow-T model to process the latent variable z l Perform forward calculation to get z p =g(z l ), and z p The distribution of The PrimaryFlow module uses the CausalTransformer structure as the basic structure for implementing Flow; The third processing unit is used for the Decoder module of the DFlow-T model according to the latent variable z l Reconstruct the training text data x to obtain training reconstructed text data The Decoder module uses a multi-layer CausalTransformer structure as the basic structure for implementing Flow; A loss function construction unit is used to construct the loss function of the DFlow-T model, wherein the loss function includes an input loss function based on maximizing likelihood estimation and a loss function of the training text data x and the training reconstruction text data obtained based on cross entropy calculation. The reconstruction loss function between A parameter updating unit is used to update the parameters of the AuxiliaryFlow module, the Primary Flow module and the Decoder module of the DFlow-T model based on the loss function until the loss function converges to obtain the trained DFlow-T model.

9. A model training device for random text generation, characterized in that: The device comprises a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the model training method for random text generation described in any one of claims 1-4 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program code, and the program code is used to execute the model training method for random text generation described in any one of claims 1-4.

Images