Training sample set generation method, and training method and device of deep generation model

Abstract

Provided are a training sample set generation method, a deep generative model training method and device. In the training sample set generation, an original sample set is divided into a plurality of aggregated sample sets with weak randomness, and a training sample set is constructed based on the aggregated sample sets. In the model training, the training sample set is used as a first training sample set; the deep generative model is iteratively trained based on the training sample set and a preset mode until a preset condition is reached, and in each iteration round: an aggregated training sample set is sampled from the training sample set; a corresponding third sample set is generated by sampling from the deep generative model based on each training sample in the aggregated training sample set; and a maximum mean difference between the aggregated training sample set and the third sample set is used as a loss function for optimization to update parameters of the deep generative model. Thus, in the model training, a method of first division and then matching is adopted for the sample set, and the model generation effect is better.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of neural network technology, and more specifically, the embodiments of the present invention relate to a training sample set generation method, a training method and apparatus for a deep generative model. Background Technology

[0002] This section is intended to provide background or context for embodiments of the invention as set forth in the claims. The description herein is not an admission that it is prior art simply because it is included in this section.

[0003] Deep generative models have achieved great success in modeling uncertainty and have been widely used in tasks such as image generation, missing data imputation, and transformational learning. Summary of the Invention

[0004] In this context, embodiments of the present invention aim to provide a method for generating training sample sets, a method for training deep generative models, and an apparatus.

[0005] In a first aspect of the present invention, a method for generating a training sample set is provided, comprising:

[0006] Provides an original sample set including multiple samples;

[0007] The original sample set is divided based on the specific association information of each sample that can represent randomness. Samples with the same or similar specific association information are grouped into the same set to obtain multiple aggregated sample sets. Each aggregated sample set is a proper subset of the original sample set.

[0008] A training sample set is constructed based on the multiple aggregated sample sets.

[0009] In one embodiment of this implementation, the original sample set is divided based on specific association information that represents randomness for each sample, and samples with the same or similar specific association information are grouped into the same set to obtain multiple aggregated sample sets, including:

[0010] Any random variable is obtained by sampling from a plurality of pre-prepared random variables;

[0011] Based on the random variable, multiple samples are sampled from the original sample set and divided into the same set to obtain an aggregated sample set;

[0012] Repeat the above steps until the preset termination condition is met. When the preset termination condition is met, multiple aggregated sample sets are obtained.

[0013] In one embodiment of this implementation, the preset termination condition includes one of the following:

[0014] Sample all random variables;

[0015] All samples are assigned to a certain set;

[0016] Repeat the preset number of times.

[0017] In one embodiment of this implementation, the original sample set is divided based on specific association information that represents randomness for each sample, and samples with the same or similar specific association information are grouped into the same set to obtain multiple aggregated sample sets, including:

[0018] Obtain the hidden layer features of each sample;

[0019] Samples with the same or similar hidden layer features are grouped into the same set to obtain multiple aggregated sample sets.

[0020] In one embodiment of this implementation, each sample is processed using a specific neural network model to obtain the hidden layer features of each sample.

[0021] In one embodiment of this implementation, the specific neural network model includes one of an encoder-decoder model and a clustering model.

[0022] In one embodiment of this implementation, when processing the original sample set based on the encoder-decoder model, the encoder is used to process the original sample set to obtain the hidden layer features of each sample.

[0023] In a second aspect of the present invention, a method for training a deep generative model is provided, comprising:

[0024] The training sample set is provided as the first training sample set using any of the methods in the first aspect;

[0025] The deep generative model is iteratively trained based on the training sample set and a preset method until a preset condition is met. In each iteration:

[0026] An aggregated training sample set is sampled from the training sample set;

[0027] A third sample set is generated from each training sample in the aggregated training sample set by the deep generation model.

[0028] The maximum mean difference between the aggregated training sample set and the third sample set is used as the loss function for optimization to update the parameters of the deep generative model.

[0029] In one embodiment of this implementation, a corresponding third sample set is generated by sampling from the deep generation model based on each training sample in the aggregated training sample set, including:

[0030] Specific association information of each training sample in the aggregated sample set is obtained one by one;

[0031] Using specific association information of each training sample and randomly sampled variables as input, the deep generative model generates and outputs the corresponding third sample set.

[0032] In one embodiment of this implementation, when the value space of the specific association information of each training sample is continuous, the conditional maximum mean difference is used as the loss function for optimization.

[0033] In one embodiment of this implementation, when calculating the maximum mean difference of conditions, the kernel function of the sample space is a mixture of 7 RBF kernel functions with bandwidths of 1, 4, 8, 16, 24, 32, and 64, respectively; the kernel function of the condition information is an RBF kernel function with a bandwidth of 1.

[0034] In one embodiment of this implementation, the deep generative model includes a preset number of deconvolutional layers, wherein the preset number is at least three.

[0035] In one embodiment of this implementation, the number of samples in one of the aggregated training sample sets is 64.

[0036] In one embodiment of this implementation, the dimension of the randomly sampled variable is 2.

[0037] In a third aspect of the present invention, a training sample set generation apparatus is provided, comprising:

[0038] The module is configured to provide an original sample set that includes multiple samples;

[0039] The partitioning module is configured to partition the original sample set based on specific association information that can represent randomness for each sample, and to group samples with the same or similar specific association information into the same set to obtain multiple aggregated sample sets, each of which is a proper subset of the original sample set.

[0040] The construction module is configured to construct a training sample set based on the multiple aggregated sample sets.

[0041] In one embodiment of this implementation, the partitioning module includes:

[0042] The sampling unit is configured to sample any random variable from a plurality of pre-prepared random variables;

[0043] The partitioning unit is configured to sample multiple samples from the original sample set based on the random variable and partition them into the same set to obtain an aggregated sample set;

[0044] The partitioning module is configured to repeatedly execute the above steps through the sampling unit and the partitioning unit until a preset termination condition is met, and when the preset termination condition is met, multiple aggregated sample sets are obtained.

[0045] In one embodiment of this implementation, the preset termination condition includes one of the following:

[0046] Sample all random variables;

[0047] All samples are assigned to a certain set;

[0048] Repeat the preset number of times.

[0049] In one embodiment of this implementation, the partitioning module includes:

[0050] The acquisition unit is configured to acquire the hidden layer features of each sample;

[0051] An aggregation unit is configured to group samples with the same or similar hidden layer features into the same set, resulting in multiple aggregated sample sets.

[0052] In one embodiment of this implementation, the acquisition unit is further configured to process each sample using a specific neural network model to obtain the hidden layer features of each sample.

[0053] In one embodiment of this implementation, the specific neural network model includes one of an encoder-decoder model and a clustering model.

[0054] In one embodiment of this implementation, the acquisition unit is further configured to use the encoder to process the original sample set when processing the original sample set based on the encoder-decoder model, so as to obtain the hidden layer features of each sample.

[0055] In a third aspect of the present invention, a training apparatus for a deep generative model is provided, comprising:

[0056] The training data providing module is configured to provide a training sample set as a first training sample set using the means of any of the fourth aspects;

[0057] The iteration module is configured to iteratively train the deep generative model based on the training sample set and a preset method until a preset condition is met. In each iteration round:

[0058] An aggregated training sample set is sampled from the training sample set;

[0059] A third sample set is generated from each training sample in the aggregated training sample set by the deep generation model.

[0060] The maximum mean difference between the aggregated training sample set and the third sample set is used as the loss function for optimization to update the parameters of the deep generative model.

[0061] In one embodiment of this implementation, a corresponding third sample set is generated by sampling from the deep generation model based on each training sample in the aggregated training sample set, including:

[0062] Specific association information of each training sample in the aggregated sample set is obtained one by one;

[0063] Using specific association information of each training sample and randomly sampled variables as input, the deep generative model generates and outputs the corresponding third sample set.

[0064] In one embodiment of this implementation, when the value space of the specific association information of each training sample is continuous, the conditional maximum mean difference is used as the loss function for optimization.

[0065] In one embodiment of this implementation, when calculating the maximum mean difference of conditions, the kernel function of the sample space is a mixture of 7 RBF kernel functions with bandwidths of 1, 4, 8, 16, 24, 32, and 64, respectively; the kernel function of the condition information is an RBF kernel function with a bandwidth of 1.

[0066] In one embodiment of this implementation, the deep generative model includes a preset number of deconvolutional layers, wherein the preset number is at least three.

[0067] In one embodiment of this implementation, the number of samples in one of the aggregated training sample sets is 64.

[0068] In one embodiment of this implementation, the dimension of the randomly sampled variable is 2.

[0069] In a fifth aspect of the present invention, a storage medium is provided that stores a computer program, which, when executed by a processor, can implement the training sample set generation method and the deep generative model training method.

[0070] In a sixth aspect of the present invention, a computing device is provided, comprising: a processor; a memory for storing processor-executable instructions; the processor being configured to execute the training sample set generation method and the deep generative model training method described above.

[0071] According to the training sample set generation method, deep generative model training method, and apparatus of the present invention, an original sample set including multiple samples is provided; the original sample set is divided based on specific correlation information that can represent randomness of each sample, and samples with the same or similar specific correlation information are grouped into the same set to obtain multiple aggregated sample sets, each aggregated sample set being a proper subset of the original sample set; a training sample set is constructed based on the multiple aggregated sample sets. Then, the training sample set is provided as a first training sample set; the deep generative model is iteratively trained based on the training sample set and a preset method until a preset condition is met. In each iteration round: an aggregated training sample set is sampled from the training sample set; a corresponding third sample set is generated by sampling from the deep generative model based on each training sample in the aggregated training sample set; the maximum mean difference between the aggregated training sample set and the third sample set is used as a loss function for optimization to update the parameters of the deep generative model. The complex probability distribution partitioning and divide-and-conquer training method in this technical solution can be applied to some training tasks using neural networks. Therefore, the technical solution of the present invention can reduce the variance of statistics during model training without increasing the sample size of the small batch, and significantly improve the sample generation effect of the deep generative model. Attached Figure Description

[0072] The above and other objects, features, and advantages of exemplary embodiments of the present invention will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. Several embodiments of the invention are illustrated in the drawings by way of example, not limitation, in which:

[0073] Figure 1 This is a schematic diagram illustrating the training of a deep generative model based on MMD.

[0074] Figure 2 This is a schematic diagram of the GMMN training process;

[0075] Figure 3 The performance of GMMN on the MNIST dataset;

[0076] Figure 4 This is a flowchart illustrating a training sample set generation method according to an embodiment of the present invention.

[0077] Figure 5 This is a flowchart illustrating a training method for a deep generative model according to an embodiment of the present invention.

[0078] Figure 6 This is a schematic diagram of the network structure of a deep generative model provided in an embodiment of the present invention;

[0079] Figure 7This is a schematic diagram illustrating the effect of VAE-based partitioning of the original sample set according to an embodiment of the present invention;

[0080] Figure 8 A schematic diagram comparing the generation quality of GMMN and GMMN-DP on different datasets in the experiments designed for this invention;

[0081] Figure 9 This is a schematic diagram of the interpolation generation results of GMMN-DP in the experiment designed in this invention;

[0082] Figure 10 This is a schematic diagram of a training sample set generation device provided in an embodiment of the present invention;

[0083] Figure 11 A schematic diagram of a training device for a deep generative model provided in an embodiment of the present invention;

[0084] Figure 12 A schematic diagram of a computer-readable storage medium provided in an embodiment of the present invention;

[0085] Figure 13 This is a schematic diagram of a computing device provided in an embodiment of the present invention.

[0086] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0087] The principles and spirit of the invention will now be described with reference to several exemplary embodiments. It should be understood that these embodiments are given merely to enable those skilled in the art to better understand and implement the invention, and are not intended to limit the scope of the invention in any way. Rather, these embodiments are provided to make this disclosure more thorough and complete, and to fully convey the scope of this disclosure to those skilled in the art.

[0088] Those skilled in the art will recognize that embodiments of the present invention can be implemented as a system, apparatus, device, method, or computer program product. Therefore, this disclosure can be specifically implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0089] According to embodiments of the present invention, a method for generating training sample sets, a method for training deep generative models, an apparatus, a medium, and a computing device are proposed.

[0090] Furthermore, the number of any elements in the accompanying drawings is for illustrative purposes only and not for limitation, and any naming is for distinction only and has no limiting meaning.

[0091] The following is a non-restrictive explanation of some key terms that appear in this article:

[0092]

[0093] Deep generative models employ multi-layered nonlinear mappings, i.e., deep neural networks, to model sample data. The model output is a probability distribution p(x), from which sample x is generated. Deep generative models are a natural choice for scenarios requiring modeling of probability and uncertainty, such as image generation, multimodal learning, and image completion with missing data.

[0094] A reproducing kernel Hilbert space is a special type of inner product space whose elements are a series of function mappings on some space X, accompanied by a kernel function k(x,x), (where x refers to random variables) and this kernel function satisfies the reproducing property, i.e. in is a non-empty set, It is defined in Hilbert space on, any x belongs to f(.) belongs to f(x) =<f(.),k(.,x)> H Conversely, this kernel function also determines an RKHS. Its key characteristic is that if the kernel function k satisfies universality, then there are enough functions in the RKHS to distinguish any element, for example... The elements in The probability of (including) (Elements and delta distribution in the data).

[0095] Maximum mean difference (MMD) is a tool used to calculate the difference between two probabilities. Its most classic application is in answering two-sample test questions, where two sets of samples {x1,...,xn} and {y1,...,ym} are given, where xi is independently and identically distributed from an unknown distribution P, and yi is independently and identically distributed from an unknown distribution Q. Based on these two sets of samples, the question is whether P = Q. MMD selects a family of functions in the RKHS (Relative Kinematics and Hierarchical Systems) and calculates the expected difference of probabilities on this family of functions as a quantification of the difference between the two probabilities, as shown in the following formula:

[0096]

[0097] Here, F is a family of functions, f is a function in the family of functions F, and E represents the expectation.

[0098] Conditional Maximum Mean Difference: The conditional information version of MMD. In MMD, the conditional information Y in the conditional probability P(X|Y) is not considered. However, CMMD takes the conditional information into account and can compare whether two conditional probabilities P(X|Y) and Q(X|Y) are the same.

[0099] Moment Matching Generative Network (GMMN) is a type of deep generative model. It uses the maximum mean difference as its learning objective. That is, the data X originates from an unknown distribution P; the GMMN model is Q. It can sample a batch of Y using Q, and optimize MMD(X,Y) to make Q as close to P as possible, thereby learning the true distribution of the data.

[0100] The principles and spirit of the present invention will be explained in detail below with reference to several representative embodiments. Invention Overview

[0102] The inventors discovered a learning (training) method for deep generative models based on moment matching GMMN, whose training criterion MMD has theoretically guaranteed correctness and a simple and explicit optimization objective, such as... Figure 1 As shown, it possesses unique appeal in both theoretical research and practical applications. Subsequent work has considered additional conditional information, extending it to a conditionally generated version of CGMMN, thus broadening its application scope.

[0103] However, one problem limits the application of GMMN in some complex real-world scenarios. The statistics involved in its application of MMD are all global, specifically the embedding of the global probability's reproducing kernel Hilbert space (RKHS). This generally requires a large sample complexity to provide a relatively accurate estimate. In practical applications, due to limitations in computational power and memory, the Stochastic Mini-Batch Gradient Descent (SGD) algorithm is almost the only training algorithm suitable for this model. However, because SGD uses a small batch of training data each time to estimate the statistics, it often introduces a large variance. Therefore, the original GMMN can only generate data with potentially low-dimensional structure, such as the MNIST handwritten digit dataset. For some structurally complex data, such as the CelebA color face dataset and the CIFAR-10 dataset of ten real-world images, GMMN using SGD as the training algorithm often performs poorly.

[0104] While recent attempts have focused on reducing the sample complexity required in MMD, these methods typically involve constructing new kernel functions with stronger statistical testing capabilities. This construction often involves performing an additional feature extraction step on the original data, followed by a kernel function with universal properties, such as the Gaussian RBF kernel. For example, MMG-GAN employs an adversarial training method to find a feature extractor z; to maintain the universality of the final kernel function, z is restricted to injective. Sutherland et al. directly constructed z by optimizing the statistical testing capabilities of MMD. However, these approaches often require approximations, which compromises MMD's theoretical ability to distinguish between arbitrarily different probabilities. Furthermore, the complexity of kernel function learning methods can introduce training instability. Therefore, the question of whether there are alternative methods to enable GMMN to perform well in complex scenarios naturally becomes an important one.

[0105] Extensive literature review reveals that existing approaches focus on the kernel function. Alternatively, if the model has a fixed kernel function, but the data is processed to adapt to the kernel function's testing capabilities, this can also improve model performance. For complex scenarios, although the overall probability is complex, the model can divide the training sample set into a series of subsets with lower randomness. By matching these subsets sequentially, the model can ultimately recover the overall probability. With this motivation and idea, this invention proposes techniques to improve GMMN performance.

[0106] like Figure 2 As shown, the core of GMMN is to use MMD as the loss function of the model. The model maintains a neural network G that can generate samples, and uses stochastic gradient descent to update the model parameters. The goal of GMMN is to make the distribution of samples generated by the neural network G as consistent as possible with the probability distribution behind the training sample set.

[0107] The algorithm iterates through rounds. For each round, the algorithm first samples a mini-batch of samples X' from G and simultaneously samples a mini-batch of samples X' from the training sample set; then, it calculates the model loss function based on X' and X.

[0108]

[0109] Where φ is the feature mapping function corresponding to the RKHS used behind MMD, ||*|| is the norm on RKHS, and x i Let x' represent the i-th sample in sample set X, where sample set X contains N samples. jLet X' represent the j-th sample in the sample set X', which includes M samples. Then, the gradient of the model parameters is calculated using gradient backpropagation, and finally, the model parameters are updated by an optimizer, such as Adam.

[0110] This method has significant room for improvement in practical performance because it cannot effectively handle the excessive variance caused by the stochastic gradient descent algorithm. Figure 3 As shown, this is the performance of GMMN on the MNIST dataset. a is the original sample and b is the generated sample. It is easy to see that the numbers in the generated sample have problems with blurring and missing numbers.

[0111] In the scenario of this invention, namely in the deep generative model, the distribution P of the generated samples of the matching model g With the training sample distribution P d (In this invention, it is also referred to as the true distribution).

[0112] When a moment-matching-based deep generative model, such as GMMN, uses MMD as the training objective, the computational complexity of the Gram matrix is ​​O(n^2). 2 k), where n is the sample size and k is the feature dimension of the sample. This limits the application to a large sample size in practice, such as all the data. Therefore, only the stochastic mini-batch gradient descent algorithm can be used. However, when calculating the RKHS kernel embedding of probabilities in the intermediate process, in general, its estimation accuracy (accuracy is basically the reciprocal of complexity) with the increase of sample size is O(1 / n). 2 However, in practice, the sample size of small batches is generally less than 128, which is far from sufficient to provide an accurate estimate of the embedding. Therefore, in scenarios with more complex samples, such as CelebA and CIFAR-10, GMMN often performs poorly.

[0113] Therefore, the core of using MMD as the optimization objective of a deep generative model lies in providing a probabilistic RKHS embedding estimate of the data. To address the problem of excessive estimation variance caused by small batches in the stochastic gradient descent algorithm, the main inventive points of this invention are summarized as follows:

[0114] This invention presents a novel training method using MMD as the training objective: directly matching the overall distribution P of the training samples. d Unlike other methods, this invention uses a divide-and-conquer approach for matching. For the overall distribution P of the training samples... d The randomness inherent in the sample space is generally very strong, meaning that it cannot be accurately estimated from a small sample set. However, if the model divides the sample space into sufficiently small subsets, then it can be expected that the randomness inherent in each subset is small, meaning that a small batch of samples can provide an accurate estimate.

[0115] In this invention, the original sample set is first divided to obtain multiple small-scale sample sets that can be used to train deep generative models. Assume that P... g To match P d The former can be parameterized by a neural network. First, some additional random variables Y are introduced to divide the overall sample space X into a series of subsets. Therefore, Each subset in the partition is represented by an indicator y i ~P(Y) is given. Because allowing sample overlap introduces greater flexibility, it is not necessary to assume X here. i They are disjoint. Under this partition, a two-step generation process for the random variable (X,Y) is actually introduced, where first, the indicator, that is, the conditional information Y, follows P(Y), and then this paper samples X from the conditional distribution P(X|Y), where P(X|Y=y i The sample space of ) is X i Assume P(Y) is fixed and known, and that there are two necessary assumptions for the partition:

[0116] <1> The marginal distribution of the joint probability is P(X)=∫ y P(X, dy) and the true distribution of the data P d (x) are the same.

[0117] <2> The randomness in each conditional distribution P(X|y) is weak. The term "weak" here means that the conditional distribution can be given a relatively accurate estimate of its embedding with a small sample size.

[0118] After the partitioning steps are completed, use P g (X|y) matches every P(X|y).

[0119] There are two scenarios here. One is that the value space of y is discrete and finite. In this case, the MMD training criterion can be used, with each P... g (X|y) is used to match P(X|y).

[0120] In general, the space of values ​​for y is continuous. In this case, the model cannot enumerate every possible value for y. i Therefore, a series of y values ​​are further aggregated, and CMMD is used to match the conditional probability embedding operator C. X|Y' That is, using the conditional probability P given by the model. g The operator C implied behind (X'|Y') X'|Y' The goal is to match the operators corresponding to the real data, where the value space of Y' consists of the aggregated values. It's important to note that this is not matching an embedding operator C of a global conditional probability P(X|Y).X|Y The latter is a global statistic, which often cannot be accurately estimated from a small sample. Conversely, this only matches a portion of the (X,Y) space. For example, for the generation process... The formula for the above generation process indicates that the overall space Y of the conditional information follows a sum of uniform distributions. After obtaining Y, the overall sample X follows a Gaussian distribution with mean Y, where U[2i,2i+1] represents a uniform distribution on [2i,2i+1], K is the number of uniform distribution mixtures, U is a uniform distribution, and N is a Gaussian distribution. If we want to obtain the conditional probability embedding μ... X|Y =2, it is not necessary to match the global C. X|Y Instead, it can only match C. X|Y′1 , where Y′1~U(2*1,2*1+1), means that Y′i follows a uniform distribution on the interval [2,3]. Here, to highlight i=1, 2 is written as 2*1.

[0121] After the above matching steps, (X,Y) can be sampled through two steps: Y ~ P(Y) and X|Y ~ P(Y). g (X|Y) is obtained. Discarding the conditional information, we can obtain X~P. d (X), which means the true distribution P of the training samples. d (x) has been restored.

[0122] The theoretical guarantee of the above matching process is as follows: in the training samples If the underlying distribution is P, then the assumption is... Satisfaction, that is Regarding the expectation of P, the result should be finite, where k(x,x) is the kernel function on RKHS; and given the continuity of the conditional information sample space, for any Y' and condition Satisfy, among which Let k represent the corresponding RKHS. Then, when the kernel function k is universal, the true sample distribution P... d (x) can be P g (x) matches.

[0123] Exemplary methods

[0124] The following is for reference. Figure 4 and Figure 5This document describes a method for generating a training sample set and a method for training a deep generative model according to exemplary embodiments of the present invention. It should be noted that the above application scenarios are shown only to facilitate understanding of the spirit and principles of the present invention, and the embodiments of the present invention are not limited in any way. Rather, the embodiments of the present invention can be applied to any applicable scenario.

[0125] refer to Figure 4 A first aspect of the present invention provides a method for generating a training sample set, comprising:

[0126] Step S110: Provide an original sample set including multiple samples;

[0127] The samples in the original sample set have a relatively complex dimensional structure, such as color face images and real photographs.

[0128] Step S120: The original sample set is divided based on the specific association information of each sample that can represent randomness. Samples with the same or similar specific association information are divided into the same set to obtain multiple aggregated sample sets. Each aggregated sample set is a proper subset of the original sample set.

[0129] In one embodiment of this implementation, step S120 includes:

[0130] Any random variable is obtained by sampling from a plurality of pre-prepared random variables;

[0131] Based on the random variable, multiple samples are sampled from the original sample set and divided into the same set to obtain an aggregated sample set;

[0132] Repeat the above steps until the preset termination condition is met. When the preset termination condition is met, multiple aggregated sample sets are obtained.

[0133] As mentioned earlier, in this embodiment, some additional random variables Y are first introduced to divide the overall training sample space X into a series of subsets, thereby having Each subset in the partition is represented by an indicator y i ~P(Y) is given.

[0134] In one embodiment of this implementation, the preset termination condition includes one of the following:

[0135] Sample all random variables;

[0136] All samples are assigned to a certain set;

[0137] Repeat the preset number of times.

[0138] It is understood that the purpose of dividing the original sample set in this embodiment is to obtain multiple aggregated sample sets with a smaller number of samples and weaker randomness. Therefore, when implementing this technical solution, those skilled in the art can set a preset termination condition according to the actual situation. For example, in a scenario where 10 aggregated sample sets are needed for deep generative model training, the preset termination condition can be set to divide 10 times.

[0139] In another embodiment of this implementation, step S120 includes:

[0140] Obtain the hidden layer features of each sample;

[0141] Samples with the same or similar hidden layer features are grouped into the same set to obtain multiple aggregated sample sets.

[0142] In one embodiment of this implementation, each sample is processed using a specific neural network model to obtain the hidden layer features of each sample.

[0143] Specifically, a pre-trained model can be introduced, which can assign a meaningful hidden layer encoding y to a sample x. This embodiment requires similar samples to have similar hidden layer encodings y. Specifically, this embodiment constructs an inverse mapping. Use it to randomly assign some hidden layer codes y to each xi. i For example, in practical applications, each x i Only one y is assigned i To form a non-overlapping partition. Having the same hidden layer encoding y i Sample x i The data is grouped into the same set, which naturally constitutes a partition of the overall data. The randomness of each group is determined by the similarity of the samples within the group and the number of samples. For example, groups with small sample sizes and high sample similarity have weak randomness. The quality of this partitioning method depends on the pre-trained model, the diversity of the samples themselves, and the randomness of the mapping M. For example, if the mapping M has no randomness, that is, it is a deterministic mapping, for example... Then all samples are divided into the same subset; for example, when the precision of mapping M is only two decimal places, the samples are more likely to be grouped together than the more precise mapping.

[0144] The selection of the pre-trained mapping model M is flexible and important. In one embodiment of this implementation, the specific neural network model includes either an encoder-decoder model or a clustering model.

[0145] <1> Encoder-decoder structure models: The encoder in this type of model can capture the latent structural information of the samples and meets the partitioning properties required above, making it a reasonable choice for this method. The encoder itself can be viewed as a mapping M, such as denoising autoencoders (DAEs), VAEs, and their variants.

[0146] <2> Clustering models aim to discover associations between similar data. Some clustering models, while performing clustering, can also provide a meaningful hidden representation of the data. For example, visual models can find the nearest neighbors of a sample in the dataset and provide hidden representations of those with close proximity. Therefore, such models are suitable as pre-training mappings for this method.

[0147] When there is additional information, such as supervision signals for some sample categories, it can be taken into account when constructing the mapping M in order to obtain a more informative and compact representation, that is, to use fewer parameters and obtain better representation results.

[0148] It is understandable that the hidden layer features, hidden layer encoding, and hidden layer representation mentioned above refer to the same concept.

[0149] In one embodiment of this implementation, when processing the original sample set based on the encoder-decoder model, the encoder is used to process the original sample set to obtain the hidden layer features of each sample.

[0150] Step S130: Construct a training sample set based on the multiple aggregated sample sets.

[0151] After partitioning the original sample set using the method described above, a training sample set for training the deep generative model can be obtained. This allows the deep generative model to be trained using the existing SGD method based on the training sample set. Furthermore, because the sample size and randomness of each aggregated sample set in the training sample set are reduced during partitioning, the variance of the model's statistical calculations can be effectively reduced during training, saving computational resources and time.

[0152] In a second aspect of the present invention, a training method for a deep generative model is provided, referring to... Figure 5 ,include:

[0153] Step S210: Use any one of the methods in the first aspect to provide a training sample set as the first training sample set;

[0154] Step S220: Based on the training sample set and the preset method, iteratively train the deep generative model until the preset conditions are met. In each iteration round:

[0155] Step S230: Sample an aggregated training sample set from the training sample set;

[0156] Step S240: Based on each training sample in the aggregated training sample set, sample from the deep generation model to generate a corresponding third sample set;

[0157] Step S250: The maximum mean difference between the aggregated training sample set and the third sample set is used as a loss function for optimization to update the parameters of the deep generative model.

[0158] In one embodiment of this implementation, a corresponding third sample set is generated by sampling from the deep generation model based on each training sample in the aggregated training sample set, including:

[0159] Specific association information of each training sample in the aggregated sample set is obtained one by one;

[0160] Using specific association information of each training sample and randomly sampled variables as input, the deep generative model generates and outputs the corresponding third sample set.

[0161] In one embodiment of this implementation, when the value space of the specific association information of each training sample is continuous, the conditional maximum mean difference is used as the loss function for optimization.

[0162] Specifically, assuming we now have a sample partitioning... It is given by a pre-given mapping M, where each sample x i They all have a hidden layer code y i .

[0163] In general, the value space of y is continuous. Under this setting, the deep generative model describes a conditional generation process, whose generator is controlled by a neural network g. θ Parameterized by parameter θ. Neural network g θ The input consists of two parts: the first part is the hidden layer code y of the sample, which serves to indicate the conditional information for sample generation; the second part consists of some additional random variables z, such as samples with a uniform distribution U[0,1] on a pre-specified dimension. An exemplary structure of the deep generative model is as follows: Figure 6 As shown.

[0164] Based on the example of the above assumptions, we first sample Y ~ P(Y), and then sample X|Y. The mapping step... Assign a y to each sample i Note that for a finite sample D, theoretically the model recovers the empirical distribution of the data, which intuitively would mean yi Viewed as a series of "anchors," these anchor the weights generated by the model samples. To describe the entire continuous hidden layer space, the algorithm can use KDE (kernel density estimator) to give the overall distribution of points in the space, while y... i Consider it as its sample, that is The above formula describes the generation process, sample y i It follows a distribution P(Y), where P(Y) is a series of Gaussian distributions laid out over certain fixed points, where y j Let N represent the sample variables in the generation process, K(·) represent the total number of generated samples, h represent the kernel function, and h be the kernel bandwidth. To generate new samples, the algorithm can first sample y from the KDE, and then from P... g Sampling in (X|y).

[0165] It is uncommon for the value space of Y to be discrete and finite, as this implies a very small dataset and a low-dimensional latent space representation. In this case, the algorithm can still use the network structure of the deep generative model, but the training objective is replaced by MMD to sequentially match P(X|y). i This can also be viewed as matching a series of distributions in an amortized manner.

[0166] In one embodiment of this implementation, when calculating the maximum mean difference of conditions, the kernel function of the sample space is a mixture of 7 RBF kernel functions with bandwidths of 1, 4, 8, 16, 24, 32, and 64, respectively; the kernel function of the condition information is an RBF kernel function with a bandwidth of 1.

[0167] In one embodiment of this implementation, the deep generative model includes a preset number of deconvolutional layers, wherein the preset number is at least three.

[0168] In one embodiment of this implementation, the number of samples in one of the aggregated training sample sets is 64.

[0169] In one embodiment of this implementation, the dimension of the randomly sampled variable is 2.

[0170] To verify the effectiveness of the technical solution of this invention, the inventors conducted experiments on the benchmark datasets MNIST and CelebA. Specifically, they used the original VAE as a pre-trained model to partition the original dataset, i.e., its encoder was used as the mapping function. In VAE, both the encoder and decoder networks use an MLP architecture with four layers. The encoder's output dimension, i.e., the hidden layer encoding dimension ny of the sample, is also determined by the dataset complexity. For MNIST, ny is set to 16 in this experiment; for CelebA, ny is set to 64. The pre-trained model in the experiment is relatively small and therefore lacks the ability to reconstruct the dataset model. For each sample x... i This experiment assigned its hidden layer code y i After that, similar hidden layer codes are no longer aggregated. Therefore, the dataset partitioning result is highly likely that each subset actually includes only one sample, and experiments have shown that this setting can accelerate convergence. Figure 6 The image shows the comparison between the original sample and the reconstructed sample after partitioning. The images in the upper layer are the reconstructed samples after partitioning, and the images in the lower layer are the original images corresponding to the images in the upper layer.

[0171] The technical solution proposed in this invention can be implemented as a neural network model GMMN-DP. The generator network follows the structure of DCGAN, consisting of 3 or 5 layers, the specific number determined by the dataset. Each layer is a transposed convolution operation, followed by batch normalization and ReLU processing. Understandably, the overall model size depends on the complexity of the dataset.

[0172] For the sample space, this experiment uses a mixture of seven RBF kernel functions, i.e. Where bandwidth σ i The values ​​are {1, 4, 8, 16, 24, 32, 64}. For conditional information, the kernel function K(y, y') is an RBF kernel function with bandwidth σ = 1. The model in this paper uses Adam as the optimizer, with a learning rate of 0.001 and other hyperparameters β = (0.9, 0.999). The number of samples in the mini-batch is B = 64, and this setting is used for experiments on all datasets. The additional randomness z dimension is set to 2. The output range of the pre-trained model VAE is -3 to 3, and the kernel bandwidth of KDE is set to h = 0.3.

[0173] The experiment compares GMMN-DP with the original GMMN. The structure, model size, and hyperparameter settings of GMMN are consistent with this embodiment. The experimental results are presented by... Figure 7As can be clearly seen, on the MNIST dataset, the GMMN-DP model can produce samples with clear boundaries, which are visually indistinguishable from the real dataset. On the CelebA dataset, the GMMN-DP model can depict many details of the face, while the GMMN-DP model can only produce a general outline.

[0174] To verify that our GMMN-DP model can fully utilize the similarity between different samples to characterize a continuous conditional generation process, rather than simply replicating the training dataset, this experiment also includes the hidden layer encoding y of the samples. i Linear interpolation is performed to generate new samples under these conditions. Specifically, a conditional variable y, given by the VAE mapping, is first fixed. Then, linear interpolation is performed on one dimension of y in the range [-3, 3] while keeping the other dimensions unchanged. The results of this experiment are shown in... Figure 8 It can be clearly observed that, in both the MNIST and CelebA datasets, as the hidden layer encoding changes continuously, the samples also change in a continuous manner, maintaining high sample quality. This demonstrates the effectiveness of the conditional probability embedding operator C modeled by CMMN-DP. X|Y' It can generalize to unseen data.

[0175] Exemplary device

[0176] After introducing the method of exemplary embodiments of the present invention, the following references are made. Figure 10 and Figure 11 A training sample set generation method and a training apparatus for a deep generative model, according to exemplary embodiments of the present invention, will be described.

[0177] Reference Figure 10 In an embodiment of the present invention, a training sample set generation apparatus is provided, comprising:

[0178] Module 310 is configured to provide an original sample set including multiple samples;

[0179] The partitioning module 320 is configured to partition the original sample set based on specific association information that can represent randomness for each sample, and to partition samples with the same or similar specific association information into the same set to obtain multiple aggregated sample sets, each of which is a proper subset of the original sample set.

[0180] The construction module 330 is configured to construct a training sample set based on the plurality of aggregated sample sets.

[0181] In one embodiment of this implementation, the partitioning module 320 includes:

[0182] The sampling unit is configured to sample any random variable from a plurality of pre-prepared random variables;

[0183] The partitioning unit is configured to sample multiple samples from the original sample set based on the random variable and partition them into the same set to obtain an aggregated sample set;

[0184] The partitioning module is configured to repeatedly execute the above steps through the sampling unit and the partitioning unit until a preset termination condition is met, and when the preset termination condition is met, multiple aggregated sample sets are obtained.

[0185] In one embodiment of this implementation, the preset termination condition includes one of the following:

[0186] Sample all random variables;

[0187] All samples are assigned to a certain set;

[0188] Repeat the preset number of times.

[0189] In one embodiment of this implementation, the partitioning module 320 includes:

[0190] The acquisition unit is configured to acquire the hidden layer features of each sample;

[0191] An aggregation unit is configured to group samples with the same or similar hidden layer features into the same set, resulting in multiple aggregated sample sets.

[0192] In one embodiment of this implementation, the acquisition unit is further configured to process each sample using a specific neural network model to obtain the hidden layer features of each sample.

[0193] In one embodiment of this implementation, the specific neural network model includes one of an encoder-decoder model and a clustering model.

[0194] In one embodiment of this implementation, the acquisition unit is further configured to use the encoder to process the original sample set when processing the original sample set based on the encoder-decoder model, so as to obtain the hidden layer features of each sample.

[0195] Reference Figure 11 In another aspect of the embodiments of the present invention, a training apparatus for a deep generative model is provided, comprising:

[0196] The training data providing module 410 is configured to provide a training sample set as a first training sample set using the means of any of the fourth aspects;

[0197] Iteration module 420 is configured to iteratively train the deep generative model based on the training sample set and a preset method until a preset condition is met. In each iteration round:

[0198] An aggregated training sample set is sampled from the training sample set;

[0199] A third sample set is generated from each training sample in the aggregated training sample set by the deep generation model.

[0200] The maximum mean difference between the aggregated training sample set and the third sample set is used as the loss function for optimization to update the parameters of the deep generative model.

[0201] In one embodiment of this implementation, a corresponding third sample set is generated by sampling from the deep generation model based on each training sample in the aggregated training sample set, including:

[0202] Specific association information of each training sample in the aggregated sample set is obtained one by one;

[0203] Using specific association information of each training sample and randomly sampled variables as input, the deep generative model generates and outputs the corresponding third sample set.

[0204] In one embodiment of this implementation, when the value space of the specific association information of each training sample is continuous, the conditional maximum mean difference is used as the loss function for optimization.

[0205] In one embodiment of this implementation, when calculating the maximum mean difference of conditions, the kernel function of the sample space is a mixture of 7 RBF kernel functions with bandwidths of 1, 4, 8, 16, 24, 32, and 64, respectively; the kernel function of the condition information is an RBF kernel function with a bandwidth of 1.

[0206] In one embodiment of this implementation, the deep generative model includes a preset number of deconvolutional layers, wherein the preset number is at least three.

[0207] In one embodiment of this implementation, the number of samples in one of the aggregated training sample sets is 64.

[0208] In one embodiment of this implementation, the dimension of the randomly sampled variable is 2.

[0209] Exemplary media

[0210] After introducing the methods and apparatus of exemplary embodiments of the present invention, the following references are made. Figure 12 A computer-readable storage medium according to exemplary embodiments of the present invention will be described, please refer to... Figure 12 The computer-readable storage medium shown is an optical disc 50, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it implements the steps described in the above method implementation, such as: providing an original sample set including multiple samples; dividing the original sample set based on specific association information that can represent randomness of each sample, grouping samples with the same or similar specific association information into the same set to obtain multiple aggregated sample sets, each aggregated sample set being a proper subset of the original sample set; and constructing a training sample set based on the multiple aggregated sample sets. Alternatively, a training sample set is provided as a first training sample set; the deep generative model is iteratively trained based on the training sample set and a preset method until a preset condition is reached. In each iteration round: an aggregated training sample set is sampled from the training sample set; a corresponding third sample set is generated by sampling from the deep generative model based on each training sample in the aggregated training sample set; the maximum mean difference between the aggregated training sample set and the third sample set is used as a loss function for optimization to update the parameters of the deep generative model; the specific implementation of each step will not be repeated here.

[0211] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0212] Exemplary computing device

[0213] After introducing the methods, apparatus, and media of exemplary embodiments of the present invention, the following references are made. Figure 13 A computing device for training sample generation and / or deep generative model training according to an exemplary embodiment of the present invention.

[0214] Figure 13 A block diagram is shown of an exemplary computing device 60 suitable for implementing embodiments of the present invention. The computing device 60 may be a computer system or a server. Figure 13 The computing device 60 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0215] like Figure 13As shown, the components of computing device 60 may include, but are not limited to: one or more processors or processing units 601, system memory 602, and bus 603 connecting different system components (including system memory 602 and processing unit 601).

[0216] The computing device 60 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computing device 60, including volatile and non-volatile media, removable and non-removable media.

[0217] System memory 602 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 6021 and / or cache memory 6022. Computing device 60 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, ROM 6023 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 13 Not shown in the image (usually referred to as a "hard drive"). Although not shown in Figure 13 The diagram illustrates that disk drives for reading and writing to removable non-volatile disks (e.g., "floppy disks") and optical disc drives for reading and writing to removable non-volatile optical discs (e.g., CD-ROMs, DVD-ROMs, or other optical media) can be provided. In these cases, each drive can be connected to bus 603 via one or more sample media interfaces. System memory 602 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0218] A program / utility 6025 having a set (at least one) of program modules 6024 may be stored, for example, in system memory 602, and such program modules 6024 include, but are not limited to, operating systems, one or more application programs, other program modules, and program samples, each or some combination of these examples may include an implementation of a network environment. Program modules 6024 typically perform the functions and / or methods described in the embodiments of the present invention.

[0219] The computing device 60 can also communicate with one or more external devices 604 (such as a keyboard, pointing device, display, etc.). This communication can be performed via the input / output (I / O) interface 605. Furthermore, the computing device 60 can also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 606. Figure 13 As shown, network adapter 606 communicates with other modules of computing device 60 (such as processing unit 601) via bus 603. It should be understood that, although... Figure 13 Other hardware and / or software modules may be used in conjunction with computing device 60, as not shown in the diagram.

[0220] The processing unit 601 executes various functional applications and sample processing by running programs stored in the system memory 602. For example, it provides an original sample set including multiple samples; it partitions the original sample set based on specific correlation information that can represent randomness for each sample, grouping samples with the same or similar specific correlation information into the same set to obtain multiple aggregated sample sets, each aggregated sample set being a proper subset of the original sample set; and it constructs a training sample set based on the multiple aggregated sample sets. Alternatively, it provides a training sample set as a first training sample set; iteratively trains the deep generative model based on the training sample set and a preset method until a preset condition is met. In each iteration round: it samples an aggregated training sample set from the training sample set; it samples a corresponding third sample set from the deep generative model based on each training sample in the aggregated training sample set; and it optimizes the parameters of the deep generative model by using the maximum mean difference between the aggregated training sample set and the third sample set as a loss function. The specific implementation methods of each step will not be repeated here. It should be noted that although the training sample set generation method and several units / modules or sub-units / sub-modules of the training device for the deep generative model are mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0221] In the description of this invention, it should be noted that the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0222] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0223] In the several embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the coupling or direct coupling or communication connection shown or discussed may be through some communication interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0224] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0225] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0226] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0227] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0228] Furthermore, although the operations of the method of the present invention are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0229] Based on the above, this application provides at least the following technical solutions:

[0230] 1. A method for generating a training sample set, comprising:

[0231] Provides an original sample set including multiple samples;

[0232] The original sample set is divided based on the specific association information of each sample that can represent randomness. Samples with the same or similar specific association information are grouped into the same set to obtain multiple aggregated sample sets. Each aggregated sample set is a proper subset of the original sample set.

[0233] A training sample set is constructed based on the multiple aggregated sample sets.

[0234] 2. The training sample set generation method as described in Scheme 1, wherein the original sample set is divided based on specific association information that can represent randomness of each sample, and samples with the same or similar specific association information are grouped into the same set to obtain multiple aggregated sample sets, including:

[0235] Any random variable is obtained by sampling from a plurality of pre-prepared random variables;

[0236] Based on the random variable, multiple samples are sampled from the original sample set and divided into the same set to obtain an aggregated sample set;

[0237] Repeat the above steps until the preset termination condition is met. When the preset termination condition is met, multiple aggregated sample sets are obtained.

[0238] 3. The training sample set generation method as described in Scheme 2, wherein the preset termination condition includes one of the following:

[0239] Sample all random variables;

[0240] All samples are assigned to a certain set;

[0241] Repeat the preset number of times.

[0242] 4. The training sample set generation method as described in Scheme 1, wherein the original sample set is divided based on specific association information that can represent randomness of each sample, and samples with the same or similar specific association information are grouped into the same set to obtain multiple aggregated sample sets, including:

[0243] Obtain the hidden layer features of each sample;

[0244] Samples with the same or similar hidden layer features are grouped into the same set to obtain multiple aggregated sample sets.

[0245] 5. The training sample set generation method as described in Scheme 2, wherein each sample is processed by a specific neural network model to obtain the hidden layer features of each sample.

[0246] 6. The training sample set generation method as described in Scheme 5, wherein the specific neural network model includes one of an encoder-decoder model and a clustering model.

[0247] 7. The training sample set generation method as described in Scheme 6, wherein when processing the original sample set based on the encoder-decoder model, the encoder is used to process the original sample set to obtain the hidden layer features of each sample.

[0248] 8. A method for training a deep generative model, comprising:

[0249] The training sample set is provided as the first training sample set using any one of the methods in schemes 1-7;

[0250] The deep generative model is iteratively trained based on the training sample set and a preset method until a preset condition is met. In each iteration:

[0251] An aggregated training sample set is sampled from the training sample set;

[0252] A third sample set is generated from each training sample in the aggregated training sample set by the deep generation model.

[0253] The maximum mean difference between the aggregated training sample set and the third sample set is used as the loss function for optimization to update the parameters of the deep generative model.

[0254] 9. The training method for the deep generative model as described in Scheme 8, wherein generating a corresponding third sample set from the deep generative model based on each training sample in the aggregated training sample set includes:

[0255] Specific association information of each training sample in the aggregated sample set is obtained one by one;

[0256] Using specific association information of each training sample and randomly sampled variables as input, the deep generative model generates and outputs the corresponding third sample set.

[0257] 10. The training method for the deep generative model as described in Scheme 9, wherein when the value space of the specific association information of each training sample is continuous, the conditional maximum mean difference is used as the loss function for optimization.

[0258] 11. The training method for the deep generative model as described in Scheme 10, wherein, when calculating the conditional maximum mean difference, the kernel function of the sample space adopts a mixture of 7 RBF kernel functions with bandwidths of 1, 4, 8, 16, 24, 32, and 64 respectively; the kernel function of the conditional information adopts an RBF kernel function with a bandwidth of 1.

[0259] 12. A training method for a deep generative model as described in any one of Schemes 8-11, wherein the deep generative model includes a preset number of deconvolutional layers, the preset number being at least three.

[0260] 13. The training method for a deep generative model as described in any one of Schemes 8-11, wherein the number of samples in one of the aggregated training sample sets is 64.

[0261] 14. The training method for a deep generative model as described in any one of Schemes 8-11, wherein the dimension of the randomly sampled variables is 2.

[0262] 15. A training sample set generation device, comprising:

[0263] The module is configured to provide an original sample set that includes multiple samples;

[0264] The partitioning module is configured to partition the original sample set based on specific association information that can represent randomness for each sample, and to group samples with the same or similar specific association information into the same set to obtain multiple aggregated sample sets, each of which is a proper subset of the original sample set.

[0265] The construction module is configured to construct a training sample set based on the multiple aggregated sample sets.

[0266] 16. The training sample set generation apparatus as described in Scheme 15, wherein the partitioning module comprises:

[0267] The sampling unit is configured to sample any random variable from a plurality of pre-prepared random variables;

[0268] The partitioning unit is configured to sample multiple samples from the original sample set based on the random variable and partition them into the same set to obtain an aggregated sample set;

[0269] The partitioning module is configured to repeatedly execute the above steps through the sampling unit and the partitioning unit until a preset termination condition is met, and when the preset termination condition is met, multiple aggregated sample sets are obtained.

[0270] 17. The training sample set generation apparatus as described in Scheme 16, wherein the preset termination condition includes one of the following:

[0271] Sample all random variables;

[0272] All samples are assigned to a certain set;

[0273] Repeat the preset number of times.

[0274] 18. The training sample set generation apparatus as described in Scheme 15, wherein the partitioning module comprises:

[0275] The acquisition unit is configured to acquire the hidden layer features of each sample;

[0276] An aggregation unit is configured to group samples with the same or similar hidden layer features into the same set, resulting in multiple aggregated sample sets.

[0277] 19. The training sample set generation apparatus as described in Scheme 18, wherein the acquisition unit is further configured to process each sample through a specific neural network model to obtain the hidden layer features of each sample.

[0278] 20. The training sample set generation apparatus as described in Scheme 19, wherein the specific neural network model includes one of an encoder-decoder model and a clustering model.

[0279] 21. The training sample set generation apparatus as described in Scheme 20, wherein the acquisition unit is further configured to process the original sample set using the encoder when processing the original sample set based on the encoder-decoder model, thereby obtaining the hidden layer features of each sample.

[0280] 22. A training device for a deep generative model, comprising:

[0281] The training data providing module is configured to provide a training sample set as a first training sample set using the apparatus of any one of schemes 15-21;

[0282] The iteration module is configured to iteratively train the deep generative model based on the training sample set and a preset method until a preset condition is met. In each iteration round:

[0283] An aggregated training sample set is sampled from the training sample set;

[0284] A third sample set is generated from each training sample in the aggregated training sample set by the deep generation model.

[0285] The maximum mean difference between the aggregated training sample set and the third sample set is used as the loss function for optimization to update the parameters of the deep generative model.

[0286] 23. The training apparatus for the deep generative model as described in Scheme 22, wherein generating a corresponding third sample set from the deep generative model based on each training sample in the aggregated training sample set includes:

[0287] Specific association information of each training sample in the aggregated sample set is obtained one by one;

[0288] Using specific association information of each training sample and randomly sampled variables as input, the deep generative model generates and outputs the corresponding third sample set.

[0289] 24. The training apparatus for the deep generative model as described in Scheme 23, wherein when the value space of the specific association information of each training sample is continuous, the conditional maximum mean difference is used as the loss function for optimization.

[0290] 25. The training device for the deep generative model as described in Scheme 24, wherein, when calculating the conditional maximum mean difference, the kernel function of the sample space adopts a mixture of 7 RBF kernel functions with bandwidths of 1, 4, 8, 16, 24, 32, and 64, respectively; the kernel function of the conditional information adopts an RBF kernel function with a bandwidth of 1.

[0291] 26. A training apparatus for a deep generative model as described in any one of claims 22-25, wherein the deep generative model includes a preset number of deconvolutional layers, the preset number being at least three.

[0292] 27. A training apparatus for a deep generative model as described in any one of claims 22-25, wherein the number of samples in one of the aggregated training sample sets is 64.

[0293] 28. A training apparatus for a deep generative model as described in any one of Schemes 22-25, wherein the dimension of the randomly sampled variables is 2.

[0294] 29. A computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to perform the method of any one of schemes 1 to 14.

[0295] 30. A computing device, comprising: a processor and a storage device;

[0296] The storage device stores a computer program that, when executed by the processor, performs the method described in any one of schemes 1 to 14.

Claims

1. A training sample set generation method, applied in the field of image generation, the method comprising: Provide an original sample set comprising multiple samples, the original sample set having a complex dimensional structure, the original sample set including color face images and real photographs; Based on specific association information that can represent randomness between each color face image and a real photograph, the color face images and real photographs are divided into groups. Color face images and real photographs with the same or similar specific association information are grouped into the same set to obtain multiple aggregated sample sets. Each aggregated sample set is a proper subset of the original sample set. A training sample set is constructed based on the multiple aggregated sample sets; The original sample set is divided based on specific association information that represents randomness for each sample. Samples with the same or similar specific association information are grouped into the same set, resulting in multiple aggregated sample sets, including: Any random variable is obtained by sampling from a plurality of pre-prepared random variables; Based on the random variable, multiple samples are sampled from the original sample set and divided into the same set to obtain an aggregated sample set; Repeat the above steps until the preset termination condition is met. When the preset termination condition is met, multiple aggregated sample sets are obtained. The preset termination condition includes one of the following: Sample all random variables; All samples are assigned to a certain set; Repeat the preset number of times; Obtain the hidden layer features of each sample; Samples with the same or similar hidden layer features are grouped into the same set to obtain multiple aggregated sample sets.

2. The training sample set generation method as described in claim 1, wherein, By processing each sample using a specific neural network model, the hidden layer features of each sample are obtained.

3. The training sample set generation method as described in claim 2, wherein, The specific neural network model includes one of the encoder-decoder model and the clustering model.

4. The training sample set generation method as described in claim 3, wherein, When processing the original sample set based on the encoder-decoder model, the encoder is used to process the original sample set to obtain the hidden layer features of each sample.

5. A training method for a deep generative model, comprising: The training sample set is provided as the first training sample set using the method of any one of claims 1-4; The deep generative model is iteratively trained based on the training sample set and a preset method until a preset condition is met. In each iteration: An aggregated training sample set is sampled from the training sample set; A third sample set is generated from each training sample in the aggregated training sample set by the deep generation model. The maximum mean difference between the aggregated training sample set and the third sample set is used as the loss function for optimization to update the parameters of the deep generative model.

6. The training method for the deep generative model as described in claim 5, wherein, Based on each training sample in the aggregated training sample set, a corresponding third sample set is generated from the deep generative model, including: Specific association information of each training sample in the aggregated sample set is obtained one by one; Using specific association information of each training sample and randomly sampled variables as input, the deep generative model generates and outputs the corresponding third sample set.

7. The training method for the deep generative model as described in claim 6, wherein, When the value space of the specific association information of each training sample is continuous, the conditional maximum mean difference is used as the loss function for optimization.

8. The training method for the deep generative model as described in claim 7, wherein, When calculating the conditional maximum mean difference, the kernel function for the sample space is a mixture of seven RBF kernel functions with bandwidths of 1, 4, 8, 16, 24, 32, and 64, respectively; the kernel function for the conditional information is an RBF kernel function with a bandwidth of 1.

9. The training method for a deep generative model as described in any one of claims 5-8, wherein, The deep generative model includes a preset number of deconvolutional layers, and the preset number is at least three.

10. The training method for a deep generative model as described in any one of claims 5-8, wherein, The number of samples in one of the aggregated training sample sets is 64.

11. The training method for a deep generative model as described in any one of claims 5-8, wherein, The dimension of the randomly sampled variables is 2.

12. A training sample set generation apparatus, used in the field of image generation, the apparatus comprising: A providing module is configured to provide an original sample set comprising multiple samples, the original sample set having a complex dimensional structure, the original sample set including color face images and real photographs; The segmentation module is configured to segment the color face images and real photos based on specific association information that can represent randomness, and to group color face images and real photos with the same or similar specific association information into the same set to obtain multiple aggregated sample sets, each of which is a proper subset of the original sample set. The construction module is configured to construct a training sample set based on the multiple aggregated sample sets; The partitioning module includes: The sampling unit is configured to sample any random variable from a plurality of pre-prepared random variables; The partitioning unit is configured to sample multiple samples from the original sample set based on the random variable and partition them into the same set to obtain an aggregated sample set; The partitioning module is configured to repeatedly execute the above steps through the sampling unit and the partitioning unit until a preset termination condition is met, and when the preset termination condition is met, multiple aggregated sample sets are obtained. The preset termination condition includes one of the following: Sample all random variables; All samples are assigned to a certain set; Repeat the preset number of times; The acquisition unit is configured to acquire the hidden layer features of each sample; An aggregation unit is configured to group samples with the same or similar hidden layer features into the same set, resulting in multiple aggregated sample sets.

13. The training sample set generation apparatus as described in claim 12, wherein, The acquisition unit is further configured to process each sample using a specific neural network model to obtain the hidden layer features of each sample.

14. The training sample set generation apparatus as described in claim 13, wherein, The specific neural network model includes one of the encoder-decoder model and the clustering model.

15. The training sample set generation apparatus as described in claim 14, wherein, The acquisition unit is further configured to use the encoder to process the original sample set when processing the original sample set based on the encoder-decoder model, so as to obtain the hidden layer features of each sample.

16. A training device for a deep generative model, comprising: The training data providing module is configured to provide a training sample set as a first training sample set using the means of any one of claims 12-15; The iteration module is configured to iteratively train the deep generative model based on the training sample set and a preset method until a preset condition is met. In each iteration round: An aggregated training sample set is sampled from the training sample set; A third sample set is generated from each training sample in the aggregated training sample set by the deep generation model. The maximum mean difference between the aggregated training sample set and the third sample set is used as the loss function for optimization to update the parameters of the deep generative model.

17. The training apparatus for a deep generative model as described in claim 16, wherein, Based on each training sample in the aggregated training sample set, a corresponding third sample set is generated from the deep generative model, including: Specific association information of each training sample in the aggregated sample set is obtained one by one; Using specific association information of each training sample and randomly sampled variables as input, the deep generative model generates and outputs the corresponding third sample set.

18. The training apparatus for a deep generative model as described in claim 17, wherein, When the value space of the specific association information of each training sample is continuous, the conditional maximum mean difference is used as the loss function for optimization.

19. The training apparatus for a deep generative model as described in claim 18, wherein, When calculating the conditional maximum mean difference, the kernel function for the sample space is a mixture of seven RBF kernel functions with bandwidths of 1, 4, 8, 16, 24, 32, and 64, respectively; the kernel function for the conditional information is an RBF kernel function with a bandwidth of 1.

20. The training apparatus for a deep generative model as described in any one of claims 16-19, wherein, The deep generative model includes a preset number of deconvolutional layers, and the preset number is at least three.

21. The training apparatus for a deep generative model as described in any one of claims 16-19, wherein, The number of samples in one of the aggregated training sample sets is 64.

22. The training apparatus for a deep generative model as described in any one of claims 16-19, wherein, The dimension of the randomly sampled variables is 2.

23. A computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to perform the method of any one of claims 1 to 11.

24. A computing device, comprising: Processors and storage devices; The storage device stores a computer program that, when executed by the processor, performs the method according to any one of claims 1 to 11.

Images