Handwritten digit generation method based on dual discriminator weighted hybrid generative adversarial network

Abstract

The present invention discloses a method for generating handwritten digits based on a dual-discriminator weighted hybrid generative adversarial network, comprising the following steps: 1. obtaining a real handwritten digit image of a target user; 2. inputting the real handwritten digit image of the target user into a pre-trained dual-discriminator weighted hybrid generative adversarial network; and 3. processing the real handwritten digit image of the target user using the dual-discriminator weighted hybrid generative adversarial network to obtain handwritten digit data. The present invention combines the advantages of multiple generators and dual discriminators, reconstructing the objective function and designing the model structure to avoid the vanishing gradient phenomenon from the perspective of the network model structure and loss function. Combining the advantages of forward KL divergence and reverse KL divergence, the generated patterns are diversified, improving the mode collapse problem encountered in GANs.

Description

Technical Field

[0001] The present invention belongs to the technical field of handwritten digit generation methods, and specifically relates to a handwritten digit generation method based on a dual-discriminator weighted hybrid generative adversarial network. Background Art

[0002] Generative Adversarial Networks (GANs) are an adversarial learning method that has been developed in recent years. GANs consist of a generator (G) and a discriminator (D). Using game theory, the two compete with each other to find a Nash equilibrium in a continuous, non-convex problem with high-dimensional parameters. GANs have been shown to generate realistic images and are very useful in data augmentation and image completion. Their primary applications include image super-resolution reconstruction, transfer learning, and image inpainting.

[0003] But given an optimal discriminator, the generator's loss function is equivalent to minimizing the true data P data (x) and generated sample P G (z) JS divergence (Jensen-Shannon) JS (P data ||P G ). In high-dimensional space, it is difficult for two distributions to intersect. Even if they do intersect, the intersecting part is a low-dimensional manifold in the high-dimensional space with a measure of 0. The intersecting part can be ignored. At this time, the JS divergence is a constant, and the gradient vanishing problem occurs. To solve this problem, Goodfellow et al. redefined the generator's loss function as -log(D(G(z))). Although the gradient vanishing problem has been solved, there is a contradiction in the objective function between minimizing the KL divergence and maximizing the JS divergence, which makes the generator training unstable. In addition, most of the generated samples are repeated and safe samples, lacking diversity, resulting in the generated images tending to be consistent and the diversity reduced. The generated samples are scarce, resulting in a mode collapse problem.

[0004] To address the mode collapse problem of GANs, the D2GAN algorithm has been proposed. This algorithm incorporates a dual discriminator concept and attempts to leverage both KL divergence and reverse KL divergence in its objective function to balance the dynamics between the generator and the discriminator. The objective function incorporates hyperparameters α and β for two purposes: first, to stabilize model learning. By reducing α and β, the effects of -D1(G(z)) and -D2(x) on optimization are reduced, respectively. Second, by increasing α and β, the algorithm encourages minimization of KL divergence and reverse KL divergence, respectively. Similar to the GAN algorithm, while the introduction of hyperparameters can mitigate model stability and mode collapse, decreasing and increasing them conflict with each other, resulting in unstable generator training. Quan Hoang et al. proposed the Mixture Generative Adversarial Network (MGAN), a multi-generator solution. MGAN simulates the distribution of real data through a mixture of multiple generators, while minimizing the cross-entropy of samples generated by multiple generators to maximize the differences between samples generated by each generator, thereby increasing the diversity of samples generated by the generators. However, MGAN only has a single discriminator and uses only JS divergence, which is prone to the vanishing gradient problem and leads to model collapse.

[0005] D2GAN introduces a dual discriminator to address the mode collapse problem of GAN, but it does not provide clear hyperparameter guidance. While its generator can learn most distributions, some are still forgotten. MGAN, which only uses JS divergence, is prone to the vanishing gradient problem, leading to mode collapse.

[0006] The above methods are not well suited for handwritten digit generation. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to address the deficiencies in the above-mentioned prior art and provide a handwritten digit generation method based on a dual-discriminator weighted hybrid generative adversarial network. The present invention combines the advantages of multiple generators and dual discriminators, reconstructs the objective function, designs the model structure, and avoids the gradient vanishing phenomenon from the perspective of the network model structure and the loss function; combines the advantages of forward KL divergence and reverse KL divergence to diversify the generated patterns and improve the pattern collapse problem of GAN.

[0008] To solve the above technical problems, the technical solution adopted by the present invention is: a handwritten digit generation method based on a dual-discriminator weighted hybrid generative adversarial network, the method comprising the following steps:

[0009] Step 1: Obtain the target user's real handwritten digital image;

[0010] Step 2: Input the target user's real handwritten digital image into the pre-trained dual-discriminator weighted hybrid generative adversarial network; wherein the training process of the dual-discriminator weighted hybrid generative adversarial network is:

[0011] Step 201: Use the MNIST dataset as a training sample;

[0012] Step 202: Build a dual-discriminator weighted hybrid generative adversarial network model; the dual-discriminator weighted hybrid generative adversarial network model includes generators G1 to G2 arranged in parallel. 10 , and connected in parallel to the generators G1 to G 10 Discriminator D1, discriminator D2 and classifier C at the output;

[0013] Step 203: Establish loss functions for the generator, discriminator, and classifier;

[0014] Step 204: Using the training samples and the loss function, and training the dual-discriminator weighted hybrid generative adversarial network model through back propagation to obtain a trained dual-discriminator weighted hybrid generative adversarial network;

[0015] Step 3: The dual-discriminator weighted hybrid generative adversarial network processes the target user's real handwritten digital image to obtain handwritten digital data.

[0016] In the above-mentioned handwritten digit generation method based on the dual discriminator weighted hybrid generative adversarial network, the training process of the dual discriminator weighted hybrid generative adversarial network in step 2 also includes step 205, performing theoretical analysis and verification on the trained dual discriminator weighted hybrid generative adversarial network to verify that under the optimal discriminator, the generator generates real handwritten digit data by minimizing the KL divergence and reverse KL divergence between the generated data and the real data.

[0017] In the above-mentioned handwritten digit generation method based on the dual discriminator weighted hybrid generative adversarial network, when building the dual discriminator weighted hybrid generative adversarial network model in step 202, the objective function of the generator is:

[0018]

[0019] Among them, T(G k ) is the generator G k The objective function of k is a natural number from 1 to 10, x '(i) is the i-th generated sample sampled from the distribution formed by the current generator, i is a natural number from 1 to m, m is the total number of generated samples in the distribution formed by the current generator, D1(x '(i) ) is when the input is x'(i) The output of the discriminator D1, D2(x '(i) ) is when the input is x '(i) When the output of the discriminator D2, ρ is D1(x '(i) ) and the value range of ρ is 0≤ρ≤1, ω is logD2(x '(i) ) and the value range of ω is 0≤ω≤1, ρ+ω=1, β is the diversity hyperparameter, is x '(i) By the u i generators The probability of generation, u i is the generator number;

[0020] The objective function of the discriminator D1 is:

[0021]

[0022] Among them, T(D1) is the objective function of the discriminator D1, x (i) is the distribution P from the real data data The i-th sample sampled from the , i is a natural number from 1 to m; D1(x (i) ) is when the input is x (i) The output of the time discriminator D1;

[0023] The input and output expression of the discriminator D2 is:

[0024]

[0025] Among them, T(D2) is the objective function of the discriminator D2, D2(x (i) ) is when the input is x (i) The output of the time discriminator D2;

[0026] The objective function of the classifier C is:

[0027]

[0028] Where T(C) is the objective function of classifier C.

[0029] In the above-mentioned handwritten digit generation method based on a dual-discriminator weighted hybrid generative adversarial network, the loss function of the generator established in step 203 is:

[0030]

[0031] in, When x obeys the distribution formed by multiple generators p model The mathematical expectation of [*], x is the distribution formed by multiple generators pmodel Sample, D1(x) is the output of the discriminator D1 when the input is x, D2(x) is the output of the discriminator D2 when the input is x, π k is the mixing probability coefficient, When x obeys the generator G k distributed The mathematical expectation of [*], C k (x) is x generated by the generator G k Probability of generation;

[0032] The loss function of the discriminator established in step 203 is:

[0033] Loss_D=Loss_D1+Loss_D2 (F6)

[0034] Among them, Loss_D1 is the loss function of discriminator D1 and is:

[0035]

[0036] Among them, Loss_D2 is the loss function of discriminator D2 and is:

[0037]

[0038] When x follows the true data distribution P data The mathematical expectation of [*] when

[0039] The loss function of the classifier established in step 203 is:

[0040]

[0041] In the above-mentioned handwritten digit generation method based on the dual discriminator weighted hybrid generative adversarial network, the specific process of using training samples and loss functions in step 204 and training the dual discriminator weighted hybrid generative adversarial network model through back propagation to obtain the trained dual discriminator weighted hybrid generative adversarial network is as follows:

[0042] Step 2041: For generators G1 to G k , discriminator D1, discriminator D2 and classifier C are initialized;

[0043] Step 2042: training the classifier C, the discriminator D1, and the discriminator D2. The specific process is as follows:

[0044] Step 20421: Sample m samples from the MNIST dataset, represented as {x (1) 、x (2) ,...,x (m)};

[0045] Step 20422, from the current generator G1 ~ G k The m samples are sampled from the distribution formed, expressed as {x '(1) 、x '(2) ,...,x '(m)}, and record the number of the generator it comes from {u1, u2, ..., u m};

[0046] Step 20423: Calculate the loss function of classifier C;

[0047] Step 20424: Calculate the loss function of the discriminator D1;

[0048] Step 20425: Calculate the loss function of the discriminator D2;

[0049] Step 20426: Update the parameters of the discriminator D1 through back propagation of the Adam optimizer to maximize the loss function of the discriminator D1 and make the discriminator D1 optimal;

[0050] Step 20427: Update the parameters of the discriminator D2 through back propagation of the Adam optimizer to maximize the loss function of the discriminator D2 and make the discriminator D2 optimal;

[0051] Step 20428: Update the parameters of the discriminator C through back propagation of the Adam optimizer to minimize the loss function of the classifier C and make the classifier C optimal;

[0052] Step 2043: Train generators G1 to G k The specific process is:

[0053] Step 20431, from the current generator G1 ~ G k The m samples are sampled from the distribution formed, expressed as {x '(1) 、x '(2) ,...,x '(m)}, and record the number of the generator it comes from {u1, u2, ..., u m};

[0054] Step 20432: Calculate the loss function of the generator;

[0055] Step 20433, update the parameters of the generator through the Adam optimizer back propagation to minimize the generator G1~G k The loss function makes the generator G1~G k Best.

[0056] In the above-mentioned handwritten digit generation method based on the dual-discriminator weighted hybrid generative adversarial network, the trained dual-discriminator weighted hybrid generative adversarial network is theoretically analyzed and verified in step 205. It is verified that under the optimal discriminator, the specific process of the generator generating real handwritten digit data by minimizing the KL divergence and reverse KL divergence between the generated data and the real data is as follows:

[0057] Step 2051: Objective function analysis of the dual-discriminator weighted hybrid generative adversarial network:

[0058] The objective function of the dual discriminator weighted hybrid generative adversarial network is:

[0059]

[0060] in, is the forward objective function, is the inverse objective function, is the classification loss; the ρ′ hyperparameter is the weight of the forward objective function and the value range of ρ′ is 0≤ρ′≤1, the ω′ hyperparameter is the weight of the reverse objective function and the value range of ω′ is 0≤ω′≤1, ρ′+ω′=1; β is the diversity hyperparameter and β>0, K is the number of generators; π k is the mixing probability coefficient, When x follows the true data distribution P data When [*] is the mathematical expectation, D1(x) is the output of the discriminator D1 when the input is x, When x obeys the distribution formed by multiple generators p mode When [*] is the mathematical expectation, D2(x) is the output of the discriminator D2 when the input is x, When x obeys the generator G k distributed The mathematical expectation of [*], C k (x) is x generated by the generator G k Probability of generation;

[0061] Step 2052: Hyperparameter significance analysis:

[0062] Step 20521: By adjusting ρ′ and ω′, the weighted sum of the KL divergence and the reverse KL divergence can be controlled, so that the complementary properties of the KL divergence and the reverse KL divergence can be better utilized to learn the real data distribution;

[0063] When ρ′=1 and ω′=0, only the generator and discriminator of the dual-discriminator weighted hybrid generative adversarial network are considered, and its objective function becomes a forward objective function, that is,

[0064]

[0065] The optimal discriminator is

[0066]

[0067] p data (x) is the real data distribution p data The probability density function, p model (x) is the distribution p formed by multiple generators model The probability density function of

[0068] Based on the optimal discriminator, the optimal generator objective function is

[0069]

[0070] KL(p data ||p model ) is the real data distribution p data and the distribution p formed by multiple generators model KL divergence between ;

[0071] In this case, the optimization of the network is equivalent to the optimization of the KL divergence, which is conducive to generating multimodal distributions but may produce potential bad samples;

[0072] When ρ′=0 and ω′=1, only the generator and discriminator of the dual-discriminator weighted hybrid generative adversarial network are considered, and its objective function becomes the inverse objective function, that is,

[0073]

[0074] The optimal discriminator is

[0075]

[0076] Based on the optimal discriminator, the optimal generator objective function is

[0077]

[0078] KL(p model ||p data ) is the real data distribution p data and the distribution p formed by multiple generators model The reverse KL divergence between

[0079] In this case, the optimization of the network is equivalent to the optimization of the reverse KL divergence, which is conducive to better capturing a single mode, but some modes may be lost, resulting in mode collapse;

[0080] When the dual-discriminator weighted hybrid generative adversarial network has both a forward objective function and a reverse objective function, that is, ρ′+ω′=1 and 0<ρ′, ω′<1, the network optimization is equivalent to optimizing the weighted sum of KL divergence and reverse KL divergence. The complementary properties of KL divergence and reverse KL divergence can better learn the real data distribution;

[0081] Step 20522: Control the classification loss by setting the value of β Impact on generator training: When β is large, the classification loss has a greater impact on the generator, and the generator learns more distribution patterns, but some overfitting-like situations may occur; when β is small, the classification loss has a smaller impact on the generator, and the generator learns fewer distribution patterns, but some underfitting-like situations may occur;

[0082] Step 2053: Theoretical analysis is performed on the dual-discriminator weighted hybrid generative adversarial network to prove that there is an optimal discriminator and classifier, and under the optimal discriminator and classifier, the generator can minimize the distribution p formed by multiple generators. model With the real data distribution p data The KL divergence and reverse KL divergence between the two to learn the real data distribution;

[0083] Theory 1: Fixed generators G1, G2, ... G K and their mixing probability coefficients π1, π2, ... π K , by maximizing and minimizing the objective function J(G 1：K , C, D1, D2) to get the optimal discriminator and the optimal classifier C * :

[0084]

[0085]

[0086]

[0087] prove:

[0088] The objective function of the dual discriminator weighted hybrid generative adversarial network can be transformed into:

[0089]

[0090] Given x, find the optimal discriminator and So that J(G 1：K , C, D1, D2) is the largest, that is

[0091]

[0092]

[0093] J(G 1：K , D1, D2) and find the partial derivatives of D1 and D2 and make them equal to 0:

[0094]

[0095]

[0096] For variables D1 and D2, the second derivative and is non-positive, so the optimal solution can be obtained, that is, the optimal discriminator

[0097] Given x, find the optimal classifier C * The process is to make the classification loss The smallest process, when the classifier C is optimal, the classifier can classify well. Classification loss is 0;

[0098] Theory 2: Given the best discriminator and the optimal classifier C * , in Nash equilibrium, if and only if p model =p data hour, Reached minimum value.

[0099]

[0100]

[0101] prove:

[0102] The optimal discriminator and the best classifier C * Substitute the objective function J(G 1：K , C, D1, D2), we can get:

[0103]

[0104] Since ρ′+ω′=1,

[0105]

[0106] Because p data and p model KL divergence between KL(p data ||p model ), and pdata and p model The reverse KL divergence KL(p model ||p data ) is always non-negative if and only if p model =p data When their values ​​are 0, Reaching the minimum value, that is, the generator reaches the optimal value, the generator can learn the entire distribution of real data, and the generator can generate real handwritten digital data.

[0107] Compared with the prior art, the present invention has the following advantages:

[0108] 1. Compared to D2GAN, this invention differs in network structure, objective function, and concept. D2GAN, with a single generator, can only learn a limited data distribution, making it prone to mode collapse. In contrast, D2WMGAN uses a joint distribution formed by multiple generators to approximate the true data distribution, effectively addressing this problem. D2WMGAN also incorporates a weighting concept, using hyperparameters ρ and ω to weight the properties of KL divergence and inverse KL divergence to avoid generating singleton samples and unseen samples. D2GAN, on the other hand, introduces hyperparameters α and β to stabilize the model, controlling the impact of KL divergence and inverse KL divergence on optimization by reducing the influence of -D1(G(z)) and -D2(x). The two algorithms have different constraints. The D2WMGAN algorithm incorporates a weighted approach with the constraints ρ = 1, ω = 0 and ρ = 0, ω = 1, taking into account the completeness of the focus. The D2GAN algorithm, with the constraints 0 < α, β ≤ 1, lacks interpretability.

[0109] 2. Compared with MGAN, the present invention has a different network structure, objective function, and concept. MGAN has only one discriminator and only uses JS divergence, which is prone to the gradient vanishing problem and thus leads to mode collapse. D2WMGAN uses two discriminators and combines KL divergence and reverse KL divergence into a unified objective function, which can solve both the gradient vanishing problem and the mode collapse problem.

[0110] 3. The optimized objective function of the present invention can degenerate to forward and reverse KL divergences. The D2WMGAN algorithm can degenerate to KL divergence and reverse KL divergence in the two extreme cases of ρ = 1, ω = 0 and ρ = 0, ω = 1, which helps to achieve multi-mode generation or capture a single mode and has strong interpretability. However, D2GAN only requires 0 < α, β ≤ 1. The objective function itself determines that it cannot degenerate to KL and reverse KL divergences, which lacks interpretability.

[0111] 4. The results after optimization of the present invention are different; when P G =P dataWhen , the optimal discriminator in the D2WGAN algorithm is The optimal discriminator in D2GAN is D1 * (x) = α, D2 * (x) = β. When optimizing the generator based on the optimal discriminator, the optimal generator is to minimize p data and p model Maximize the JS divergence between generators when comparing the forward KL and reverse KL divergences between them. That is, the generator gradually learns the entire distribution of the real data, if and only if the generator learns the entire distribution of the real data, that is, p model =p data When p data and p model The KL divergence and reverse KL divergence between them are 0.

[0112] 5. In response to the technical problems existing in GAN, D2GAN and MGAN, the present invention proposes a handwritten digit generation method based on a dual-discriminator weighted hybrid generative adversarial network. The present invention combines the advantages of multiple generators and dual discriminators, reconstructs the objective function, designs the model structure, and avoids the gradient vanishing phenomenon from the perspective of network model structure and loss function; combines the advantages of forward KL divergence and reverse KL divergence to diversify the generated patterns and improve the pattern collapse problem of GAN.

[0113] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0114] Figure 1 This is a flowchart of the method of Embodiment 1 and Embodiment 2 of the present invention;

[0115] Figure 2 Schematic diagram of the structure of the dual-discriminator weighted hybrid generative adversarial network in Examples 1 and 2 of the present invention;

[0116] Figure 3 The maximum and minimum loss and non-saturation loss graphs of the GAN network;

[0117] Figure 4 is the linear loss graph of the GAN network;

[0118] Figure 5 This is a handwritten digit image generated by the GAN network using the MNIST dataset;

[0119] Figure 6 The effect diagram of handwritten digital images generated by the D2GAN network using the MNIST dataset;

[0120] Figure 7This is a handwritten digit image generated by the MGAN network using the MNIST dataset;

[0121] Figure 8 This is the rendering of a handwritten digital image generated by the D2WMGAN network of the present invention using the MNIST dataset. DETAILED DESCRIPTION

[0122] The following is a clear and complete description of the technical solutions of the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0123] To address the problem of mode collapse in GAN and its poor ability to generate diverse samples, this paper proposes a dual-discriminator weighted hybrid generative adversarial network, which introduces multiple generators, dual discriminators, and weighted ideas to construct a new network structure and objective function, avoiding the phenomenon of gradient vanishing from the perspective of network model structure and loss function; combining the advantages of forward KL divergence and reverse KL divergence to diversify the generated patterns, thereby avoiding the problem of mode collapse.

[0124] In GAN, two loss functions, log(1-D(G(z))) and -log(D(G(z))), are proposed to train the generator G. However, both loss functions have the problem of gradient disappearance. Figure 3 As shown in , in the early stage of training, the generator G is very poor, and the generated samples are very different from the real samples. The discriminator D can easily identify fake samples, and its output value D(G(z)) tends to 0. The gradient of log(1-D(G(z))) is small and cannot provide enough gradient for training the generator, and -log(D(G(z))) will be saturated at the beginning of training; in the later stage of training, the generator G gradually gets better, and the difference between the generated samples and the real samples becomes smaller. The discriminator D cannot easily identify fake samples that can be mistaken for the real ones, and its output value D(G(z)) tends to 1. The gradient of log(1- D(G(z))) becomes larger and larger and saturates, while the gradient of -log(D(G(z))) loss function becomes smaller and smaller and tends to zero. Therefore, both log(1-D(G(z))) and -log(D(G(z))) loss functions will have gradient disappearance problems at the beginning and end of training; as shown in Figure 4As shown, the derivative of the linear loss function -D(G(z)) is -1, and its gradient remains constant throughout the training process, effectively alleviating the vanishing gradient phenomenon. By leveraging the linear relationship of the linear loss function -D(G(z)), the stochastic gradient descent method can converge quickly and achieve faster computation speed. Practice has shown that using the linear loss function -D(·) to train the generator produces more diverse samples.

[0125] Example 1

[0126] like Figure 1 As shown, the handwritten digit generation method based on the dual-discriminator weighted hybrid generative adversarial network of the present invention comprises the following steps:

[0127] Step 1: Obtain the target user's real handwritten digital image;

[0128] Step 2: Input the target user's real handwritten digit image into the pre-trained dual discriminator weighted hybrid generative adversarial network (D2WMGAN network); wherein the training process of the dual discriminator weighted hybrid generative adversarial network (D2WMGAN network) is as follows:

[0129] Step 201: Use the MNIST dataset as a training sample;

[0130] The MNIST dataset consists of handwritten digits from 250 different people, 50% of whom are high school students and 50% are Census Bureau staff. The dataset contains 70,000 images of handwritten digits, 60,000 of which are training sets and 10,000 are test sets. The images are divided into 10 categories, 0-9, and are all normalized to 28*28 grayscale images.

[0131] Step 202: Build a dual discriminator weighted hybrid generative adversarial network model (D2WMGAN network model); the dual discriminator weighted hybrid generative adversarial network model (D2WMGAN network model) includes generators G1 to G2 arranged in parallel. 10 , and connected in parallel to the generators G1 to G 10 The discriminator D1, discriminator D2 and classifier C at the output end; Figure 2 As shown;

[0132] Among them, generators G1~G 10 Used to generate numbers 0 to 9;

[0133] In this embodiment, when building the dual-discriminator weighted hybrid generative adversarial network model (D2WMGAN network model) in step 202, the objective function of the generator is:

[0134]

[0135] Among them, T(G k ) is the generator G k The objective function of k is a natural number from 1 to 10, x '(i) is the i-th generated sample sampled from the distribution formed by the current generator, i is a natural number from 1 to m, m is the total number of generated samples in the distribution formed by the current generator, D1(x '(i) ) is when the input is x '(i) The output of the discriminator D1, D2(x '(i) ) is when the input is x '(i) When the output of the discriminator D2, ρ is D1(x '(i) ) and the value range of ρ is 0≤ρ≤1, ω is logD2(x '(i) ) and the value range of ω is 0≤ω≤1, ρ+ω=1, β is the diversity hyperparameter, is x '(i) By the u i generators The probability of generation, u i is the generator number;

[0136] In the specific implementation, m generated samples {x (1) , x (2) ,...,x (m)}, and the generator numbers from which they come are recorded as {u1, u2, ..., u m};

[0137] In specific implementation, the impact of classification loss on generator training can be controlled by setting the size of β. When β is large, the classification loss has a greater impact on the generator, and the generator learns more distribution patterns, but some overfitting-like situations may occur; when β is small, the classification loss has a smaller impact on the generator, and the generator learns fewer distribution patterns, but some underfitting-like situations may occur;

[0138] The objective function of the discriminator D1 is:

[0139]

[0140] Among them, T(D1) is the objective function of the discriminator D1, x (i) is the distribution P from the real data data The i-th sample sampled from the source, i is a natural number from 1 to m (i.e., from the real data distribution P dataThe total number of samples sampled in is equal to the total number of samples generated in the distribution formed by the current generator); D1(x (i) ) is when the input is x (i) The output of the time discriminator D1;

[0141] The input and output expression of the discriminator D2 is:

[0142]

[0143] Among them, T(D2) is the objective function of the discriminator D2, D2(x (i) ) is when the input is x (i) The output of the time discriminator D2;

[0144] The objective function of the classifier C is:

[0145]

[0146] Where T(C) is the objective function of classifier C.

[0147] In order to compare the performance of the dual-discriminator weighted hybrid generative adversarial network with the generative adversarial network, the dual-discriminator generative adversarial network, and the hybrid generative adversarial network in generating handwritten digits, the model structure of the generator and the discriminator is very simple, with only one hidden layer;

[0148] Step 203: Establish loss functions for the generator, discriminator, and classifier;

[0149] In this embodiment, the loss function of the generator established in step 203 is:

[0150]

[0151] in, When x obeys the distribution formed by multiple generators p model The mathematical expectation of [*], x is the distribution formed by multiple generators p model Sample, D1(x) is the output of the discriminator D1 when the input is x, D2(x) is the output of the discriminator D2 when the input is x, π k is the mixing probability coefficient (i.e., the weight of the k-th generator in the mixed distribution), When x obeys the generator G k distributed The mathematical expectation of [*], C k (x) is x generated by the generator G k Probability of generation;

[0152] The loss function of the discriminator established in step 203 is:

[0153] Loss_D=Loss_D1+Loss_D2 (F6)

[0154] Among them, Loss_D1 is the loss function of discriminator D1 and is:

[0155]

[0156] Among them, Loss_D2 is the loss function of discriminator D2 and is:

[0157]

[0158] When x follows the true data distribution P data The mathematical expectation of [*] when

[0159] The loss function of the classifier established in step 203 is:

[0160]

[0161] Step 204: Using the training samples and the loss function, the dual discriminator weighted hybrid generative adversarial network model (D2WMGAN network model) is trained by back propagation to obtain a trained dual discriminator weighted hybrid generative adversarial network (D2WMGAN network);

[0162] In this embodiment, the specific process of using the training samples and loss function in step 204 and training the dual discriminator weighted hybrid generative adversarial network model (D2WMGAN network model) through back propagation to obtain the trained dual discriminator weighted hybrid generative adversarial network (D2WMGAN network) is as follows:

[0163] Step 2041: For generators G1 to G k , discriminator D1, discriminator D2 and classifier C are initialized;

[0164] Step 2042: training the classifier C, the discriminator D1, and the discriminator D2. The specific process is as follows:

[0165] Step 20421, from the MNIST dataset (real data distribution P data ) to obtain m samples, expressed as {x (1) 、x (2) ,…,x (m)};

[0166] Step 20422, from the current generator G1 ~ G k The m samples are sampled from the distribution formed, expressed as {x '(1) 、x '(2),…,x '(m)}, and record the number of the generator it comes from {u1, u2, ..., u m};

[0167] Step 20423: Calculate the loss function of classifier C;

[0168] Step 20424: Calculate the loss function of the discriminator D1;

[0169] Step 20425: Calculate the loss function of the discriminator D2;

[0170] Step 20426: Update the parameters of the discriminator D1 through back propagation of the Adam optimizer to maximize the loss function of the discriminator D1 and make the discriminator D1 optimal;

[0171] Step 20427: Update the parameters of the discriminator D2 through back propagation of the Adam optimizer to maximize the loss function of the discriminator D2 and make the discriminator D2 optimal;

[0172] Step 20428: Update the parameters of the discriminator C through back propagation of the Adam optimizer to minimize the loss function of the classifier C and make the classifier C optimal;

[0173] Step 2043: Train generators G1 to G k The specific process is:

[0174] Step 20431, from the current generator G1 ~ G k The m samples are sampled from the distribution formed, expressed as {x '(1) 、x '(2) ,…,x '(m)}, and record the number of the generator it comes from {u1, u2, ..., u m};

[0175] Step 20432: Calculate the loss function of the generator;

[0176] Step 20433, update the parameters of the generator through the Adam optimizer back propagation to minimize the generator G1~G k The loss function makes the generator G1~G k Best.

[0177] Step 3: The dual discriminator weighted hybrid generative adversarial network (D2WMGAN network) processes the real handwritten digital image of the target user to obtain handwritten digital data.

[0178] Example 2

[0179] This embodiment differs from Embodiment 1 in that the training process of the dual discriminator weighted hybrid generative adversarial network (D2WMGAN network) described in step 2 further includes step 205, performing theoretical analysis and verification on the trained dual discriminator weighted hybrid generative adversarial network (D2WMGAN network), verifying that under the optimal discriminator, the generator generates authentic handwritten digital data by minimizing the KL divergence and reverse KL divergence between the generated data and the authentic data.

[0180] In this embodiment, the trained dual-discriminator weighted hybrid generative adversarial network (D2WMGAN network) described in step 205 is theoretically analyzed and verified to verify that under the optimal discriminator, the specific process of the generator generating real handwritten digital data by minimizing the KL divergence and reverse KL divergence between the generated data and the real data is as follows:

[0181] Step 2051: Objective function analysis of the dual-discriminator weighted hybrid generative adversarial network:

[0182] The objective function of the dual discriminator weighted hybrid generative adversarial network (D2WMGAN network) is:

[0183]

[0184] in, is the forward objective function, is the inverse objective function, is the classification loss; the ρ′ hyperparameter is the weight of the forward objective function and the value range of ρ′ is 0≤ρ′≤1, the ω′ hyperparameter is the weight of the reverse objective function and the value range of ω′ is 0≤ω′≤1, p′+ω′=1; β is the diversity hyperparameter and β>0, K is the number of generators (K is 10 in this embodiment); π k is the mixing probability coefficient (i.e., the weight of the k-th generator in the mixed distribution), When x follows the true data distribution P data When [*], D1(x) is the output of the discriminator D1 when the input is x, When x obeys the distribution formed by multiple generators p model When [*] is the mathematical expectation, D2(x) is the output of the discriminator D2 when the input is x, When x obeys the generator G k distributed The mathematical expectation of [*], C k (x) is x generated by the generator G k Probability of generation;

[0185] In specific implementation, when there is no prior information, π is generally set k =1 / K and

[0186] Step 2052: Hyperparameter significance analysis:

[0187] Step 20521: By adjusting ρ′ and ω′, the weighted sum of the KL divergence and the reverse KL divergence can be controlled, so that the complementary properties of the KL divergence and the reverse KL divergence can be better utilized to learn the real data distribution;

[0188] When ρ′=1 and ω′=0, only the generator and discriminator of the dual-discriminator weighted hybrid generative adversarial network (D2WMGAN network) are considered, and its objective function becomes a forward objective function, that is,

[0189]

[0190] The optimal discriminator is

[0191]

[0192] p data (x) is the real data distribution p data The probability density function, p model (x) is the distribution p formed by multiple generators model The probability density function of

[0193] Based on the optimal discriminator, the optimal generator objective function is

[0194]

[0195] KL(p data ||P model ) is the real data distribution p data and the distribution p formed by multiple generators model KL divergence between ;

[0196] In this case, the optimization of the network is equivalent to the optimization of the KL divergence, which is conducive to generating multimodal distributions but may produce potential bad samples;

[0197] When ρ′=0 and ω′=1, only the generator and discriminator of the dual-discriminator weighted hybrid generative adversarial network (D2WMGAN network) are considered, and its objective function becomes the inverse objective function, that is,

[0198]

[0199] The optimal discriminator is

[0200]

[0201] Based on the optimal discriminator, the optimal generator objective function is

[0202]

[0203] KL(p model ||p data ) is the real data distribution p data and the distribution p formed by multiple generators model The reverse KL divergence between

[0204] In this case, the optimization of the network is equivalent to the optimization of the reverse KL divergence, which is conducive to better capturing a single mode, but some modes may be lost, resulting in mode collapse;

[0205] When the dual-discriminator weighted hybrid generative adversarial network (D2WMGAN network) has both forward and reverse objective functions, that is, ρ′+ω′=1 and 0<ρ′, ω′<1, the network optimization is equivalent to optimizing the weighted sum of KL divergence and reverse KL divergence. The complementary properties of KL divergence and reverse KL divergence can better learn the real data distribution;

[0206] Step 20522: Control the classification loss by setting the value of β Impact on generator training: When β is large, the classification loss has a greater impact on the generator, and the generator learns more distribution patterns, but some overfitting-like situations may occur; when β is small, the classification loss has a smaller impact on the generator, and the generator learns fewer distribution patterns, but some underfitting-like situations may occur;

[0207] Step 2053: Theoretical analysis is performed on the dual discriminator weighted hybrid generative adversarial network (D2WMGAN network), proving that there are optimal discriminators and classifiers, and under the optimal discriminators and classifiers, the generator can minimize the distribution p formed by multiple generators. model With the real data distribution p data The KL divergence and reverse KL divergence between the two methods are used to learn the real data distribution;

[0208] Theory 1: Fixed generators G1, G2, ... G K and their mixing probability coefficients π1, π2, ... π K , by maximizing and minimizing the objective function J(G 1：K , C, D1, D2) to get the optimal discriminator and the optimal classifier C * :

[0209]

[0210]

[0211]

[0212] prove:

[0213] The objective function of the dual discriminator weighted hybrid generative adversarial network can be transformed into:

[0214]

[0215]

[0216] Given x, find the optimal discriminator and So that J(G 1：K , C, D1, D2) is the largest, that is

[0217]

[0218]

[0219] J(G 1：K , D1, D2) and find the partial derivatives of D1 and D2 and make them equal to 0:

[0220]

[0221]

[0222] For variables D1 and D2, the second derivative and is non-positive, so the optimal solution can be obtained, that is, the optimal discriminator

[0223] Given x, find the optimal classifier C * The process is to make the classification loss The smallest process, when the classifier C is optimal, the classifier can classify well. Classification loss is 0;

[0224] Theory 2: Given the best discriminator and the optimal classifier C * , in Nash equilibrium, if and only if p model =p data hour, Reached minimum value.

[0225]

[0226]

[0227] Represents the distribution p formed by multiple generatorsmodel Equal to the true data distribution p data Any x when

[0228] prove:

[0229] The optimal discriminator and the best classifier C * Substitute the objective function J(C 1：K , C, D1, D2), we can get:

[0230]

[0231] Since ρ′+ω′=1,

[0232]

[0233] Because p data and p model KL divergence between KL(p data ||p model ), and p data and p model The reverse KL divergence KL(p model ||p data ) is always non-negative if and only if p model =p data When their values ​​are 0, Reaching the minimum value, that is, the generator reaches the optimal value, the generator can learn the entire distribution of real data, and the generator can generate real handwritten digital data.

[0234] The remaining methods are the same as in Example 1.

[0235] In order to verify the technical effects that can be produced by the present invention, the following simulation tests were conducted:

[0236] 1. Experimental environment:

[0237] The simulation experiment environment of the present invention is as follows: the processor is Inter Xeon E5-2620 v4, the operating system is 64-bit Windows 10, the graphics card is NVIDIA GeForce RTX 2080Ti, the PyCharm editor is used, the Python 3.7 version is used, and the TensorFlow deep learning framework is used; the MNIST handwritten digit image data set contains 70,000 handwritten digit images, of which 60,000 are training sets and 10,000 are test sets. This experiment only uses the 60,000 training data set.

[0238] 2. Simulation experiment content:

[0239] Handwritten digit images were generated on the original generative adversarial network, the dual-discriminator generative adversarial network, the hybrid generative adversarial network, and the dual-discriminator weighted hybrid generative adversarial network. Except for their own algorithms, the network structures are basically the same, all containing one hidden layer. The same number of iterative trainings were conducted, and the results of the generated handwritten digits were compared. The results are shown in the figure below. Figure 5 、 Figure 6 、 Figure 7 and Figure 8 shown.

[0240] 3. Analysis of simulation results:

[0241] from Figure 5 and Figure 6 It can be seen that the handwritten digits generated by GAN and D2GAN do not include all the digits from 0 to 9. The generated digits are not clear, and there is a lot of noise and some unrecognizable samples. This is because during training, GAN finds a pattern that is easy to fool the discriminator, and the probability of generating this pattern increases, resulting in a smaller number of generated handwritten digit types. D2GAN generates more types of handwritten digits, indicating that it can learn most distributions, but some distributions are still forgotten. Figure 7 and Figure 8 For the handwritten digits generated by MGAN and D2WMGAN, each row of digit samples in the figure is generated by the same generator. Figure 7 It can be seen that although MGAN can generate all numbers from 0 to 9, there are some numbers generated by individual generators with the same shape, which is a single generator mode collapse problem. Figure 8 It can be seen intuitively that D2WMGAN can generate clear and realistic handwritten digits from 0 to 9, and each generator can basically learn a digit distribution. At the same time, the handwriting styles of the same digit generated by each generator are also different, indicating that D2WMGAN has better generation diversity and verifies that the model can solve the model collapse problem.

[0242] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0243] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0244] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0245] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0246] The foregoing descriptions of specific exemplary embodiments of the present invention are for purposes of illustration and description. These descriptions are not intended to limit the invention to the precise forms disclosed, and it is apparent that many variations and modifications are possible in light of the foregoing teachings. The exemplary embodiments have been selected and described for the purpose of explaining the specific principles of the invention and their practical application, thereby enabling those skilled in the art to realize and utilize a variety of exemplary embodiments of the invention and various options and modifications. The scope of the invention is intended to be defined by the claims and their equivalents.

[0247] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any way. Any simple modification, change and equivalent structural change made to the above embodiment based on the technical essence of the present invention shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A handwritten digit generation method based on a dual-discriminator weighted hybrid generative adversarial network, characterized in that: The method comprises the following steps: Step 1: Obtain the target user's real handwritten digital image; Step 2: Input the target user's real handwritten digital image into the pre-trained dual-discriminator weighted hybrid generative adversarial network; wherein the training process of the dual-discriminator weighted hybrid generative adversarial network is: Step 201: Use the MNIST dataset as a training sample; Step 202: Build a dual-discriminator weighted hybrid generative adversarial network model; the dual-discriminator weighted hybrid generative adversarial network model includes generators G1 to G2 arranged in parallel. 10 , and connected in parallel to the generators G1 to G 10 Discriminator D1, discriminator D2 and classifier C at the output; When building the dual-discriminator weighted hybrid generative adversarial network model in step 202, the objective function of the generator is: Among them, T(G k ) is the generator G k The objective function, k is a natural number from 1 to 10, x' (i) is the i-th generated sample sampled from the distribution formed by the current generator, i is a natural number from 1 to m, m is the total number of generated samples in the distribution formed by the current generator, D1(x' (i) ) is when the input is x' (i) The output of the discriminator D1, D2(x' (i) ) is when the input is x' (i) When the output of the discriminator D2, ρ is D1(x' (i) ) and the value range of ρ is 0≤ρ≤1, ω is logD2(x' (i) ) and the value range of ω is 0≤ω≤1, ρ+ω=1, β is the diversity hyperparameter, is x' (i) By the u i generators The probability of generation, u i is the generator number; The objective function of the discriminator D1 is: Among them, T(D1) is the objective function of the discriminator D1, x (i) is the distribution P from the real data data The i-th sample sampled from the , i is a natural number from 1 to m; D1(x (i) ) is when the input is x (i) The output of the time discriminator D1; The input and output expression of the discriminator D2 is: Among them, T(D2) is the objective function of the discriminator D2, D2(x (i) ) is when the input is x (i) The output of the time discriminator D2; The objective function of the classifier C is: Where T(C) is the objective function of classifier C; Step 203: Establish loss functions for the generator, discriminator, and classifier; Step 204: Using the training samples and the loss function, and training the dual-discriminator weighted hybrid generative adversarial network model through back propagation to obtain a trained dual-discriminator weighted hybrid generative adversarial network; Step 3: The dual-discriminator weighted hybrid generative adversarial network processes the target user's real handwritten digital image to obtain handwritten digital data.

2. The handwritten digit generation method based on a dual-discriminator weighted hybrid generative adversarial network according to claim 1, characterized in that: The training process of the dual-discriminator weighted hybrid generative adversarial network described in step 2 also includes step 205, performing theoretical analysis and verification on the trained dual-discriminator weighted hybrid generative adversarial network to verify that under the optimal discriminator, the generator generates real handwritten digital data by minimizing the KL divergence and reverse KL divergence between the generated data and the real data.

3. The handwritten digit generation method based on a dual-discriminator weighted hybrid generative adversarial network according to claim 1 or 2, characterized in that: The loss function of the generator established in step 203 is: in, When x obeys the distribution formed by multiple generators p model The mathematical expectation of [*], x is the distribution formed by multiple generators p model Sample, D1(x) is the output of the discriminator D1 when the input is x, D2(x) is the output of the discriminator D2 when the input is x, π k is the mixing probability coefficient, When x obeys the generator G k distributed The mathematical expectation of [*], C k (x) is x generated by the generator G k Probability of generation; The loss function of the discriminator established in step 203 is: Loss_D=Loss_D1+Loss_D2 (F6) Among them, Loss_D1 is the loss function of discriminator D1 and is: Among them, Loss_D2 is the loss function of discriminator D2 and is: When x follows the true data distribution P data The mathematical expectation of [*] when The loss function of the classifier established in step 203 is:

4. The handwritten digit generation method based on a dual-discriminator weighted hybrid generative adversarial network according to claim 1 or 2, characterized in that: The specific process of using the training samples and loss function in step 204 and training the dual discriminator weighted hybrid generative adversarial network model through back propagation to obtain the trained dual discriminator weighted hybrid generative adversarial network is as follows: Step 2041: For generators G1 to G k , discriminator D1, discriminator D2 and classifier C are initialized; Step 2042: training the classifier C, the discriminator D1, and the discriminator D2. The specific process is as follows: Step 20421: Sample m samples from the MNIST dataset, represented as {x (1) 、x (2) ,…,x (m) }; Step 20422, from the current generator G1 ~ G k The m samples are sampled from the distribution formed, expressed as {x' (1) 、x' (2) ,…,x' (m) }, and record the number of the generator from which it comes {u1,u2,…,u m }; Step 20423: Calculate the loss function of classifier C; Step 20424: Calculate the loss function of the discriminator D1; Step 20425: Calculate the loss function of the discriminator D2; Step 20426: Update the parameters of the discriminator D1 through back propagation of the Adam optimizer to maximize the loss function of the discriminator D1 and make the discriminator D1 optimal; Step 20427: Update the parameters of the discriminator D2 through back propagation of the Adam optimizer to maximize the loss function of the discriminator D2 and make the discriminator D2 optimal; Step 20428: Update the parameters of the discriminator C through back propagation of the Adam optimizer to minimize the loss function of the classifier C and make the classifier C optimal; Step 2043: Train generators G1 to G k The specific process is: Step 20431, from the current generator G1 ~ G k The m samples are sampled from the distribution formed, expressed as {x' (1) 、x' (2) ,…,x' (m) }, and record the number of the generator from which it comes {u1,u2,…,u m }; Step 20432: Calculate the loss function of the generator; Step 20433, update the parameters of the generator through the Adam optimizer back propagation to minimize the generator G1~G k The loss function makes the generator G1~G k Best.

5. The handwritten digit generation method based on a dual-discriminator weighted hybrid generative adversarial network according to claim 2, characterized in that: Theoretical analysis and verification of the trained dual-discriminator weighted hybrid generative adversarial network in step 205 verifies that under the optimal discriminator, the specific process of the generator generating real handwritten digital data by minimizing the KL divergence and reverse KL divergence between the generated data and the real data is as follows: Step 2051: Objective function analysis of the dual-discriminator weighted hybrid generative adversarial network: The objective function of the dual discriminator weighted hybrid generative adversarial network is: in, is the forward objective function, is the inverse objective function, is the classification loss; the ρ' hyperparameter is the weight of the forward objective function and the value range of ρ' is 0≤ρ'≤1, the ω' hyperparameter is the weight of the reverse objective function and the value range of ω' is 0≤ω'≤1, ρ'+ω'=1; β is the diversity hyperparameter and β>0, K is the number of generators; π k is the mixing probability coefficient, When x follows the true data distribution P data When [*] is the mathematical expectation, D1(x) is the output of the discriminator D1 when the input is x, When x obeys the distribution formed by multiple generators p model When [*] is the mathematical expectation, D2(x) is the output of the discriminator D2 when the input is x, When x obeys the generator G k distributed The mathematical expectation of [*], C k (x) is x generated by the generator G k Probability of generation; Step 2052: Hyperparameter significance analysis: Step 20521: By adjusting ρ' and ω', the weighted sum of KL divergence and reverse KL divergence can be controlled, and the complementary characteristics of KL divergence and reverse KL divergence can be better utilized to learn the real data distribution; When ρ'=1 and ω'=0, only the generator and discriminator of the dual-discriminator weighted hybrid generative adversarial network are considered, and its objective function becomes a forward objective function, that is, The optimal discriminator is p data (x) is the real data distribution p data The probability density function, p model (x) is the distribution p formed by multiple generators model The probability density function of Based on the optimal discriminator, the optimal generator objective function is KL(p data ||p model ) is the real data distribution p data and the distribution p formed by multiple generators model KL divergence between ; In this case, the optimization of the network is equivalent to the optimization of the KL divergence, which is conducive to generating multimodal distributions but may produce potential bad samples; When ρ'=0 and ω'=1, only the generator and discriminator of the dual-discriminator weighted hybrid generative adversarial network are considered, and its objective function becomes the inverse objective function, that is, The optimal discriminator is Based on the optimal discriminator, the optimal generator objective function is KL(p model ||p data ) is the real data distribution p data and the distribution p formed by multiple generators model The reverse KL divergence between In this case, the optimization of the network is equivalent to the optimization of the reverse KL divergence, which is conducive to better capturing a single mode, but some modes may be lost, resulting in mode collapse; When the dual-discriminator weighted hybrid generative adversarial network has both a forward objective function and a reverse objective function, that is, ρ'+ω'=1 and 0<ρ',ω'<1, the network optimization is equivalent to optimizing the weighted sum of KL divergence and reverse KL divergence. The complementary properties of KL divergence and reverse KL divergence can better learn the real data distribution; Step 20522: Control the classification loss by setting the value of β Impact on generator training: When β is large, the classification loss has a greater impact on the generator, and the generator learns more distribution patterns, but some overfitting-like situations may occur; when β is small, the classification loss has a smaller impact on the generator, and the generator learns fewer distribution patterns, but some underfitting-like situations may occur; Step 2053: Theoretical analysis is performed on the dual-discriminator weighted hybrid generative adversarial network to prove that there is an optimal discriminator and classifier, and under the optimal discriminator and classifier, the generator can minimize the distribution p formed by multiple generators. model With the real data distribution p data KL divergence and reverse KL divergence between the two to learn the real data distribution; Theory 1: Fixed generators G1, G2, ... G K and their mixing probability coefficients π1, π2, ... π K , by maximizing and minimizing the objective function J(G 1:K ,C,D1,D2) to obtain the optimal discriminator and the optimal classifier C * : prove: The objective function of the dual discriminator weighted hybrid generative adversarial network can be transformed into: Given x, find the optimal discriminator and So that J(G 1:K ,C,D1,D2) is the largest, that is J(G 1:K ,D1,D2) and find the partial derivatives of D1 and D2 and make them equal to 0: For variables D1 and D2, the second derivative and is non-positive, so the optimal solution can be obtained, that is, the optimal discriminator Given x, find the optimal classifier C * The process is to make the classification loss The smallest process, when the classifier C is optimal, the classifier can classify well. Classification loss is 0; Theory 2: Given the best discriminator and the optimal classifier C * , in Nash equilibrium, if and only if p model =p data hour, Reach a minimum value; prove: The optimal discriminator and the best classifier C * Substitute the objective function J(G 1:K ,C,D1,D2), we can get: Since ρ'+ω'=1, Because p data and p model KL divergence between KL(p data ||p model ), and p data and p mode The reverse KL divergence KL(p model ||p data ) is always non-negative if and only if p model =p data When their values ​​are 0, Reaching the minimum value, that is, the generator reaches the optimal value, the generator can learn the entire distribution of real data, and the generator can generate real handwritten digital data.

Images