Unsupervised deep learning method based on synthetic data generation

Abstract

The invention belongs to the technical field of unsupervised deep learning methods, and particularly relates to an unsupervised deep learning method based on synthetic data generation, and the method comprises the steps: searching a clustering center and a standard deviation of original unlabeled data through a Gaussian mixture model; bayesian optimization selects an optimal superposition weight for a Gaussian mixture model by measuring the difference between isotropic Gaussian spot generation data based on a Gaussian mixture model clustering center and a standard deviation and original data distribution; the Gaussian mixture model determines the clustering number and the standard deviation under the optimal superposition weight, and the isotropic Gaussian spots generate a large amount of synthetic data with labels and balance based on the clustering number and the standard deviation; the synthetic data is used for training a deep neural network model; and the trained deep neural network model executes a downstream task based on the initial label-free data. The sample equalization of the generated synthetic data is realized, the training of the deep neural network is assisted, and the model precision in an unsupervised learning scene is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of unsupervised deep learning methods, and in particular, relates to an unsupervised deep learning method based on synthetic data generation. Background Art

[0002] With the development of wireless networks, mining data information and extracting its features for application have become common tasks in the data age. For example, unlabeled image classification tasks in computer vision and abnormal traffic detection tasks in networks. However, in the real world, there are various types of unlabeled data. Such data poses a huge challenge to data mining due to the lack of labels as a reference and guide for training. At present, the core idea of ​​common methods for data mining tasks on unlabeled data is to analyze the statistical characteristics and similarities of data to discover the potential structure and patterns in the data. Such methods are called unsupervised training. Among them, deep neural networks stand out among many methods due to their flexible ability to change. They can realize complex feature transformations to adapt to different unsupervised learning tasks.

[0003] Deep neural networks need a large amount of data as input samples for training to extract and summarize the rules in the data. The current mainstream solution is to generate synthetic data based on existing samples to enrich the sample size involved in training. The generation of synthetic data follows the paradigm of first analyzing the original data structure and then generating synthetic data based on the analysis results. For example, data generation methods based on principal component analysis and synthetic data generation methods based on Gaussian mixture functions. Although the above methods can solve the problem of insufficient sample size for unsupervised deep learning to a certain extent, the existing methods do not pay attention to the sample balance problem of generating synthetic data, such as the imbalance of the number of samples of different types and the low differentiation of the characteristics of samples of different types, which leads to confusion in the training process of deep neural networks, poor results in practical applications, and it is difficult to meet the needs of practical applications.

[0004] Therefore, how to improve the training process of deep neural networks in existing methods to achieve sample balance in generating synthetic data, assist in the training of deep neural networks, and improve the model accuracy in unsupervised learning scenarios is a technical problem that needs to be solved urgently. Summary of the invention

[0005] The purpose of the present invention is to provide an unsupervised deep learning method based on synthetic data generation, so as to achieve sample balance in generating synthetic data, assist in the training of deep neural networks, and improve the model accuracy in unsupervised learning scenarios.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is as follows: An unsupervised deep learning method based on synthetic data generation, comprising the following steps: S1: Gaussian mixture model obtains superposition weights from the Bayesian optimizer, finds cluster centers and standard deviations based on unlabeled raw data, and isotropic Gaussian spots generate synthetic data using the cluster centers and standard deviations of the current Gaussian mixture model; S2: measuring the distribution difference between the synthetic data and the original data, and obtaining the optimal superposition weight for the Gaussian mixture function based on Bayesian optimization iteration; S3: Isotropic Gaussian takes the cluster center and standard deviation of the Gaussian mixture model under the optimal superposition weight as the benchmark as the synthetic data generator, which generates labeled synthetic data with similar distribution to the original data; S4: inputting the labeled synthetic data into a deep neural network model for model training; S5: Input the initial unlabeled data into the trained deep neural network model to perform downstream tasks.

[0007] Preferably, the specific process of finding the cluster center and standard deviation based on the unlabeled original data in step S1 is as follows: S11: The unlabeled raw data is represented as X ,Include N samples, each sample i Include M Features, any x i,j Indicates i The sample j The value of the feature in , where 1≤ i ≤ N , 1≤ j ≤ M , the unlabeled original data X Divide K kind, K =[1, 2, ···, k ]; S11: The unlabeled raw data is represented as X ,Include N samples, each sample i Include M Features, any x i,j Indicates i The sample j The value of the feature in , where 1≤ i ≤ N , 1≤ j ≤ M , the unlabeled original data X Divide K kind, K =[1, 2, ···, k ]; S12: Unlabeled raw data X Clustering is performed, and the specific formula is as follows: ; in, is the clustering result, For sample i The class to which it belongs, , is the class to which sample 1 belongs, is the class to which sample 2 belongs, For sample N The class to which it belongs; S13: Calculate the cluster center based on the probability of the given clustering data sample C and standard deviation σ .

[0008] Preferably, in step S13, the cluster center is calculated according to the probability of the given cluster calculation data sample. C The specific process of calculating the standard deviation σ is as follows: S131: Based on K The Gaussian mixture model is created by linear superposition of Gaussian models. The specific formula is as follows: ; in, w k is the superposition parameter, , For the k The mean vector of the Gaussian components, For the k The covariance matrix of the Gaussian components, For the k The multivariate normal density function of the components; S132: Bayesian optimizer randomly initializes superposition parameters for Gaussian mixture models w , and calculate the original sample mean based on the original data , the covariance matrix , generating the sample mean , then the Gaussian mixture model has 3× k Parameters to be updated; S133: Calculate each sample i The probability of belonging to each cluster , a total of iterative calculations n × k The calculation formula is as follows: ; in, It is j The mixing weights of the components, , x represents the sample set, x i For the i samples, For the j The mean vector of the Gaussian components, For the j The covariance matrix of the Gaussian components; S134: Isotropic Gaussian Spots Generate labeled and balanced synthetic data of the same scale as the original data.

[0009] Preferably, in each iteration in step S133, the parameters are updated by the following formula: ; ; ; in, is the sample mean, , N is the total number of data samples, I is the identity matrix, is the regularization term.

[0010] Preferably, the specific process of generating labeled and balanced synthetic data with the same scale as the original data by using isotropic Gaussian spots in step S134 is as follows: S1341: Isotropic Gaussian blobs generate labels based on the optimal weighted parameters of the Gaussian mixture model: K Randomly select a cluster center from the clusters k , at this time the data object d i belong k , the generator is based on the labeled data object d i To generate tags, d i ∈ D , and obtain a size of N The included dataset D Collection of tags ; S1342: Data generator isotropic Gaussian spots according to C k , σ k is the center and standard deviation, For the sample size, generate a data object d i , the formula is as follows: ; in, Normal distribution , the generation process is executed K times until the isotropic Gaussian spots have generated samples for all clusters, and the downstream obtains a N Balanced dataset of synthetic samples D .

[0011] Preferably, the specific process of obtaining the optimal superposition weight for the Gaussian mixture function based on Bayesian optimization iteration in step S2 is as follows: S21: The Kolmogorov-Smirnov test is used to determine whether the data generated by the generator is similar to the original data distribution. The statistic S of the Kolmogorov-Smirnov test is: S= sup x |F m ( x )- G n ( x ) | ; in, sup x express |F m ( x )- G n ( x ) | Maximum value, F m ( x ) is the empirical distribution function of the original data, G n ( x ) is the empirical distribution function of synthetic data, ; ; I is the indicator function, when X i ≤ X , Y i ≤ X When I=1, n is the sample size, m represents the number of original data samples; S22: Loop through steps S12, S13, and S21 to obtain the optimal superposition weight of the Gaussian mixture model.

[0012] Preferably, the specific process of generating labeled synthetic data in step S3 is as follows: S31: Based on the optimal superposition weight of the Gaussian mixture model obtained, run the Gaussian mixture function to obtain the cluster center obtained by iterative Gaussian mixture model under this weight. C and standard deviation σ; S32: Specify the required number of samples, run the isotropic Gaussian spotting in rounds according to the number of cluster centers, and generate labeled sample balanced synthetic data D'.

[0013] Preferably, the specific process of inputting the labeled synthetic data into the deep neural network model for model training in step S4 is as follows: S41: deploying a deep neural network model, including a mask layer, a dense layer, a batch normalization layer, and a random dropout layer, wherein the mask layer is the first layer of the deep neural network model, and the dense layer is provided with a plurality of neurons and uses a linear transformation and an activation function that increases nonlinearity; S42: Compare the median of the input sequence with the preset specified value through the mask layer Equal samples are masked, the specific formula is as follows: ; in, represents the i-th element of the mask vector, Represents the i-th sample value in the input sequence; S43: Calculation is performed through dense layers. The specific formula is as follows: h=ReLU(WD'+b); Among them, h is the output after the dense layer is calculated, W is the linearly changing learnable weight matrix, b is the bias vector, ReLU is the rectified linear unit function, ReLU= max (0, z ), z It is the result of linear transformation in the neural network. When the linear unit function is modified by ReLU, if z>0, the output is z ; Otherwise, output 0, D' is the synthetic data with labeled sample balance; S44: Input the output result of the dense layer to the batch normalization layer for processing. The specific formula is as follows: ; in, and They are h The mean and standard deviation of the current batch, and are learnable parameters; S45: The output result of the batch normalization layer is input to the random dropout layer for processing. The specific formula is as follows: ; in,d is a binary masking vector, each element has a probability of being 0 p , the probability of each element being 1 is 1 / p ; S46: Output of the dropout layer Enter with k The output dense layer of neurons and softmax activation function calculates the probability that the input data belongs to each class of labels: ; The dimension of y is k ×N, is the weight matrix, is the bias vector, and the softmax activation function is: ; in, z i is the input vector z The i elements; S47: The probability results of the predicted labels are measured using cross entropy: ; in, Indicates the class to which sample i belongs, synlog indicates the synchronous logarithm, To predict the probability, the training loss is used to update the learnable parameters of the deep neural network through the Adam optimizer until the deep neural network converges.

[0014] Preferably, the specific process of step S5 is as follows: The initial unlabeled data sample X Input the deep neural network model pre-trained by synthetic data in step S4, output the corresponding results, and execute downstream tasks.

[0015] The beneficial effects of the present invention include: The present invention provides an unsupervised deep learning method based on synthetic data generation, wherein a Gaussian mixture model searches for the cluster centers and standard deviations of the original unlabeled data; Bayesian optimization selects the optimal superposition weight for the Gaussian mixture model by measuring the difference between the data generated by isotropic Gaussian spots based on the cluster centers and standard deviations of the Gaussian mixture model and the distribution of the original data; the Gaussian mixture model determines the number of clusters and the standard deviation under the optimal superposition weight, and the isotropic Gaussian spots generate a large amount of labeled and balanced synthetic data based on the number of clusters and the standard deviation; the synthetic data is used to train a deep neural network model; the trained deep neural network model performs downstream tasks based on the initial unlabeled data. According to the statistical characteristics of the unlabeled data, a large amount of labeled synthetic data similar to its distribution can be generated based on the clustering method, thereby providing sufficient training samples for the deep neural network model training in the unsupervised scenario, improving the performance of the model, achieving sample balance in generating synthetic data, assisting the training of the deep neural network, and improving the model accuracy in the unsupervised learning scenario.

[0016] The present invention mines the statistical features of unlabeled data by superimposing weights on a Bayesian optimized Gaussian mixture model, generates labeled synthetic data to expand the training sample size, and provides label references for deep neural network model training, thereby improving model performance. Considering the sample sizes of various types of synthetic data, a consistent number of samples are generated for each type of data to achieve balanced synthetic data samples and avoid confusion caused by unbalanced generated samples in deep neural network model learning. The structure of the deep neural network can be flexibly adjusted according to specific tasks, and the output support of the pre-trained deep neural network can be further processed to meet the needs of various downstream tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 Schematic diagram of a flow chart of an unsupervised deep learning method based on synthetic data generation according to the present invention.

[0018] Figure 2 The present invention is a schematic diagram of a process for selecting optimal superposition weights for a Gaussian mixture model using the Bayesian optimizer. DETAILED DESCRIPTION

[0019] The following is combined with Figure 1~Figure 2 The present invention is further described in detail: Example 1 See also Figure 1 and Figure 2 As shown, an unsupervised deep learning method based on synthetic data generation includes the following steps: S1: The Gaussian mixture model randomly obtains superposition weights from the Bayesian optimizer and finds cluster centers and standard deviations based on the unlabeled original data. The isotropic Gaussian spots use the cluster centers and standard deviations of the current Gaussian mixture model to generate synthetic data. S2: Kolmogorov-Smirnov test measures the distribution difference between the synthetic data and the original data, and obtains the optimal superposition weight for the Gaussian mixture function based on Bayesian optimization iteration; S3: Isotropic Gaussian takes the cluster centers and standard deviations of the Gaussian mixture model under the optimal superposition weights as the benchmark as a synthetic data generator, which generates labeled synthetic data with a distribution similar to the original data.

[0020] Run the Gaussian mixture function with the optimal parameters of the Gaussian mixture model, and then specify the required number of samples N' to run the isotropic Gaussian spot generation label and synthetic data. To ensure the balance of the generated data, the generator will call the cluster center and variance one by one to generate synthetic data with balanced sample sizes. The isotropic Gaussian spots are called K times. Each time a C k generate samples, of which , the downstream gets a balanced dataset of size , the clusters are represented as labels for each synthetic data.

[0021] S4: Input the labeled synthetic data into a deep neural network model for model training; in this step, an adjustable deep neural network model is designed to meet the execution needs of downstream tasks. The deep neural network includes a mask layer, a dense layer, a batch normalization layer, and a random dropout layer.

[0022] The combination of the above network layers can be flexibly adjusted for different specific tasks. The output results finally enter the output dense layer with k neurons and softmax activation function to calculate the probability that the input data belongs to each type of label. The predicted label probability results are measured by cross entropy as the training loss, and the learnable parameters of the deep neural network model are updated by the gradient descent method until the deep neural network converges.

[0023] S5: Input the initial unlabeled data into the trained deep neural network model, perform downstream tasks, and output the execution results.

[0024] Example 2 Based on Example 1, the specific process of finding the cluster center and standard deviation based on the unlabeled original data in step S1 is as follows: S11: Constructing structured data representation: Representing unlabeled raw data as X ,IncludeN samples, each sample i Include M Features, any x i,j Indicates i The sample j The value of the feature in , where 1≤ i ≤ N , 1≤ j ≤ M , the unlabeled original data X Divide K kind, K =[1, 2, ···, k ]; S12: Unlabeled raw data X Clustering is performed, and the specific formula is as follows: ; in, is the clustering result, For sample i The class to which it belongs, , is the class to which sample 1 belongs, is the class to which sample 2 belongs, For sample N The class to which it belongs; S13: Calculate the cluster center based on the probability of the given clustering data sample C and standard deviation σ ; S13: Calculate the cluster center based on the probability of the given clustering data sample C and standard deviation σ.

[0025] In this embodiment, in step S13, the cluster center is calculated according to the probability of the given cluster calculation data sample. C The specific process of calculating the standard deviation σ is as follows: S131: Based on K The Gaussian mixture model is created by linear superposition of Gaussian models. The specific formula is as follows: ; in, w k is the superposition parameter, , For the k The mean vector of the Gaussian components, For the k The covariance matrix of the Gaussian components, For the k The multivariate normal density function of the components; S132: Bayesian optimizer randomly initializes superposition parameters for Gaussian mixture models w , and calculate the original sample mean based on the original data , the covariance matrix , generating the sample mean , then the Gaussian mixture model has 3× k Parameters to be updated; S133: Calculate each sample i The probability of belonging to each cluster , a total of iterative calculations n × k The calculation formula is as follows: ; in, It is j The mixing weights of the components, , x represents the sample set, x i For the i samples, For the j The mean vector of the Gaussian components, For the j The covariance matrix of the Gaussian components; S134: Isotropic Gaussian Spots Generate labeled and balanced synthetic data of the same scale as the original data.

[0026] In each iteration in step S133, the parameters are updated by the following formula: ; ; ; in, is the sample mean, , N is the total number of data samples, I is the identity matrix, is the regularization term.

[0027] The specific process of generating labeled and balanced synthetic data with the same scale as the original data by using isotropic Gaussian spots in step S134 is as follows: S1341: Isotropic Gaussian blobs generate labels based on the optimal weighted parameters of the Gaussian mixture model: K Randomly select a cluster center from the clusters k , at this time the data object d i belong k , the generator is based on the labeled data objectd i To generate labels, d i ∈ D , and obtain a size of N The included dataset D Collection of tags ; S1342: Data generator isotropic Gaussian spots according to C k , σ k is the center and standard deviation, For the sample size, generate a data object d i , the formula is as follows: ; in, Normal distribution , the generation process is executed K times until the isotropic Gaussian spots have generated samples for all clusters, and the downstream obtains a N Balanced dataset of synthetic samples D .

[0028] Isotropic Gaussian spots generate synthetic data of the same size as the original data. At the same time, the present invention focuses on the balance of the generated synthetic data. Therefore, in the data generation process, isotropic Gaussian spots are called in turn based on the number of samples and the number of clusters to generate data with balanced sample size. Isotropic Gaussian spots use the cluster center C and standard deviation σ generated in the Gaussian mixture function as the cluster center and standard deviation, where C k , σ k They represent the cluster center and standard deviation of a specific class, respectively, and generate a data set with similar characteristics and the same size as the original data. To ensure the sample balance of the synthetic data, the isotropic Gaussian spots are called cyclically.

[0029] Example 3 Based on Example 1 or Example 2, the specific process of obtaining the optimal superposition weight for the Gaussian mixture function based on Bayesian optimization iteration in step S2 is as follows: S21: The Kolmogorov-Smirnov test is used to determine whether the data generated by the generator is similar to the original data distribution. The statistic S of the Kolmogorov-Smirnov test is: S= sup x |F m ( x )- G n (x ) | ; in, sup x express |F m ( x )- G n ( x ) | Maximum value.

[0030] F m ( x ) is the empirical distribution function of the original data, G n ( x ) is the empirical distribution function of synthetic data, ; ; I is the indicator function, when X i ≤ X , Y i ≤ X When , I = 1, otherwise it is 0, n is the sample size, m Represents the number of original data samples; S22: Loop through steps S12, S13, and S21 to obtain the optimal superposition weight of the Gaussian mixture model.

[0031] The specific process of generating labeled synthetic data in step S3 is as follows: S31: Based on the optimal superposition weight of the Gaussian mixture model obtained, run the Gaussian mixture function to obtain the cluster center C and standard deviation obtained by iterative Gaussian mixture model under this weight. σ ; S32: Specify the required number of samples, run isotropic Gaussian spots according to the number of cluster centers, and generate labeled sample balanced synthetic data .

[0032] Example 4 On the basis of Example 1 or Example 2 or Example 3, the specific process of inputting the labeled synthetic data into the deep neural network model for model training in step S4 is as follows: S41: Deploy a deep neural network model, including a mask layer, a dense layer, a batch normalization layer and a random dropout layer. The mask layer is the first layer of the deep neural network model. The dense layer is provided with multiple neurons and uses a linear transformation and a ReLU activation function that increases nonlinearity. For different specific tasks, the combination of the above network layers can be flexibly adjusted.

[0033] S42: Compare the median of the input sequence with the preset specified value through the mask layer Equal samples are masked, the specific formula is as follows: ; represents the i-th element of the mask vector, Represents the i-th sample value in the input sequence.

[0034] S43: Calculation is performed through dense layers. The specific formula is as follows: h=ReLU(WD'+b); Among them, h is the output after the dense layer is calculated, W is the linearly changing learnable weight matrix, b is the bias vector, ReLU is the rectified linear unit function, ReLU= max (0, z ), z It is the result of linear transformation in the neural network. When the linear unit function is modified by ReLU, if z>0, the output is z ; Otherwise, output 0, and D' is the synthetic data balanced with labeled samples.

[0035] S44: Input the output result of the dense layer to the batch normalization layer for processing. The normalization layer can disrupt the data order, accelerate the convergence of the network, and improve the generalization ability of the network. The specific formula is as follows: ; in, and They are h The mean and standard deviation of the current batch, and are learnable parameters; S45: The output of the batch normalization layer is input to the random dropout layer for processing. The random dropout layer can avoid the problem of overfitting of the deep neural network and reduce the training time. The specific formula is as follows: ; in, d is a binary masking vector, each element has a probability of being 0 p , the probability of each element being 1 is 1 / p ; S46: Predicting labels based on synthetic data: Output of the dropout layer Enter with k The output dense layer of neurons and softmax activation function calculates the probability that the input data belongs to each class of labels: ; The dimension of y is k ×N, is the weight matrix, is the bias vector, and the softmax activation function is: ; Where, is the input vector z The i elements, z j is the input vector z The j elements; S47: Back propagation training of deep neural network: The probability results of the predicted labels are measured by cross entropy: ; in, Indicates the class to which sample i belongs, synlog indicates the synchronous logarithm, To predict the probability, the training loss is used to update the learnable parameters of the deep neural network through the Adam optimizer until the deep neural network converges.

[0036] The specific process of step S5 is as follows: The initial unlabeled data sample X Input the deep neural network model pre-trained by synthetic data in step S4, output the corresponding results, and execute downstream tasks.

[0037] The present invention mines the statistical features of unlabeled data by superimposing weights on a Bayesian optimized Gaussian mixture model, generates labeled synthetic data to expand the training sample size, and provides label references for deep neural network model training, thereby improving model performance. Considering the sample sizes of various types of synthetic data, a consistent number of samples are generated for each type of data to achieve balanced synthetic data samples and avoid confusion caused by unbalanced generated samples in deep neural network model learning. The structure of the deep neural network of the present invention can be flexibly adjusted according to specific tasks, and the output of the pre-trained deep neural network supports further processing, which can meet the needs of various downstream tasks. It can be applicable to a variety of downstream tasks, and supports setting other data processing methods for specific problems based on the results of pre-training.

[0038] In summary, the present invention provides an unsupervised deep learning method based on synthetic data generation, wherein the Gaussian mixture model searches for the cluster center and standard deviation of the original unlabeled data; Bayesian optimization selects the optimal superposition weight for the Gaussian mixture model by measuring the difference between the isotropic Gaussian spot generated data based on the cluster center and standard deviation of the Gaussian mixture model and the distribution of the original data; the Gaussian mixture model determines the number of clusters and the standard deviation under the optimal superposition weight, and the isotropic Gaussian spot generates a large amount of labeled and balanced synthetic data based on the number of clusters and the standard deviation; the synthetic data is used to train the deep neural network model; the trained deep neural network model performs downstream tasks based on the initial unlabeled data. According to the statistical characteristics of the unlabeled data, a large amount of labeled synthetic data similar to its distribution can be generated based on the clustering method, thereby providing sufficient training samples for the deep neural network model training in the unsupervised scenario, improving the performance of the model, and by analyzing the distribution characteristics of the original data, generating synthetic data that matches and is balanced with the original data distribution through the clustering method to assist the training of the deep neural network, improving the model accuracy in the unsupervised learning scenario, and effectively solving the technical problem that the current generation of synthetic data is unbalanced and leads to low accuracy of the deep learning model.

Claims

1. An unsupervised deep learning method based on synthetic data generation, characterized in that: The following steps are involved: S1: Gaussian mixture model obtains superposition weights from the Bayesian optimizer, finds cluster centers and standard deviations based on unlabeled raw data, and isotropic Gaussian spots generate synthetic data using the cluster centers and standard deviations of the current Gaussian mixture model; S2: measuring the distribution difference between the synthetic data and the original data, and obtaining the optimal superposition weight for the Gaussian mixture function based on Bayesian optimization iteration; S3: Isotropic Gaussian takes the cluster center and standard deviation of the Gaussian mixture model under the optimal superposition weight as the benchmark as the synthetic data generator, which generates labeled synthetic data with similar distribution to the original data; S4: inputting the labeled synthetic data into a deep neural network model for model training; S5: Input the initial unlabeled data into the trained deep neural network model to perform downstream tasks.

2. The unsupervised deep learning method based on synthetic data generation according to claim 1, characterized in that: The specific process of finding the cluster center and standard deviation based on the unlabeled original data in step S1 is as follows: S11: The unlabeled raw data is represented as X ,Include N samples, each sample i Include M Features, any x i,j Indicates i The sample j The value of the feature in , where 1≤ i ≤ N , 1≤ j ≤ M , the unlabeled original data X Divide K kind, K =[1, 2, ···, k ]; S12: Unlabeled raw data X Clustering is performed, and the specific formula is as follows: ； in, is the clustering result, For sample i The class to which it belongs, , is the class to which sample 1 belongs, is the class to which sample 2 belongs, For sample N The class it belongs to; S13: Calculate the cluster center based on the probability of the given clustering data sample C and standard deviation σ.

3. The unsupervised deep learning method based on synthetic data generation according to claim 2, characterized in that: In step S13, the cluster center is calculated according to the probability of the given clustering calculation data sample. C The specific process of calculating the standard deviation σ is as follows: S131: Based on K The Gaussian mixture model is created by linear superposition of Gaussian models. The specific formula is as follows: ； in, w k is the superposition parameter, , For the k The mean vector of the Gaussian components, For the k The covariance matrix of the Gaussian components, For the k The multivariate normal density function of the components; S132: Bayesian optimizer randomly initializes superposition parameters for Gaussian mixture models w , and calculate the original sample mean based on the original data , the covariance matrix , generating the sample mean , then the Gaussian mixture model has 3× k Parameters to be updated; S133: Calculate each sample i The probability of belonging to each cluster , a total of iterative calculations n × k The calculation formula is as follows: ； in, It is j The mixing weights of the components, , x represents the sample set, x i For the i samples, For the j The mean vector of the Gaussian components, For the j The covariance matrix of the Gaussian components; S134: Isotropic Gaussian Spots Generate labeled and balanced synthetic data of the same scale as the original data.

4. The unsupervised deep learning method based on synthetic data generation according to claim 3, characterized in that: In each iteration in step S133, the parameters are updated by the following formula: ； ； ； in, is the sample mean, , N is the total number of data samples, I is the identity matrix, is the regularization term.

5. The unsupervised deep learning method based on synthetic data generation according to claim 3, characterized in that: The specific process of generating labeled and balanced synthetic data with the same scale as the original data by using isotropic Gaussian spots in step S134 is as follows: S1341: Isotropic Gaussian blobs generate labels based on the optimal weighted parameters of the Gaussian mixture model: K Randomly select a cluster center from the clusters k , at this time the data object d i belong k , the generator is based on the labeled data object d i To generate labels, d i ∈ D , and obtain a size of N The included dataset D Collection of tags ; S1342: Data generator isotropic Gaussian spots according to C k , σ k is the center and standard deviation, For the sample size, generate a data object d i , the formula is as follows: ； in, Normal distribution , the generation process is executed K times until the isotropic Gaussian spots have generated samples for all clusters, and the downstream obtains a N Balanced dataset of synthetic samples D .

6. The unsupervised deep learning method based on synthetic data generation according to claim 2, characterized in that: The specific process of obtaining the optimal superposition weight for the Gaussian mixture function based on Bayesian optimization iteration in step S2 is as follows: S21: The Kolmogorov-Smirnov test is used to determine whether the data generated by the generator is similar to the original data distribution. The statistic S of the Kolmogorov-Smirnov test is: S= sup x |F m ( x )- G n ( x ) | ； in, sup x express |F m ( x )- G n ( x ) | Maximum value, F m ( x ) is the empirical distribution function of the original data, G n ( x ) is the empirical distribution function of synthetic data, ； ； I is the indicator function, when X i ≤ X , Y i ≤ X When I=1, n is the sample size, m represents the number of original data samples; S22: Loop through steps S12, S13, and S21 to obtain the optimal superposition weight of the Gaussian mixture model.

7. The unsupervised deep learning method based on synthetic data generation according to claim 6, characterized in that: The specific process of generating labeled synthetic data in step S3 is as follows: S31: Based on the optimal superposition weight of the Gaussian mixture model obtained, run the Gaussian mixture function to obtain the cluster center obtained by iterative Gaussian mixture model under this weight. C and standard deviation σ ; S32: Specify the required number of samples, run the isotropic Gaussian spotting in rounds according to the number of cluster centers, and generate labeled sample balanced synthetic data D'.

8. The unsupervised deep learning method based on synthetic data generation according to claim 1, characterized in that: The specific process of inputting the labeled synthetic data into the deep neural network model for model training in step S4 is as follows: S41: deploying a deep neural network model, including a mask layer, a dense layer, a batch normalization layer, and a random dropout layer, wherein the mask layer is the first layer of the deep neural network model, and the dense layer is provided with a plurality of neurons and uses a linear transformation and an activation function that increases nonlinearity; S42: Compare the median of the input sequence with the preset specified value through the mask layer Equal samples are masked, the specific formula is as follows: ； in, represents the i-th element of the mask vector, Represents the i-th sample value in the input sequence; S43: Calculation is performed through dense layers. The specific formula is as follows: h=ReLU(WD'+b); Among them, h is the output after the dense layer is calculated, W is the linearly changing learnable weight matrix, b is the bias vector, ReLU is the rectified linear unit function, ReLU= max (0, z ), z It is the result of linear transformation in the neural network. When the linear unit function is modified by ReLU, if z>0, the output is z ; Otherwise, output 0, D' is the synthetic data with labeled sample balance; S44: Input the output result of the dense layer to the batch normalization layer for processing. The specific formula is as follows: ； in, and They are h The mean and standard deviation of the current batch, and are learnable parameters; S45: The output result of the batch normalization layer is input to the random dropout layer for processing. The specific formula is as follows: ； in, d is a binary masking vector, each element has a probability of being 0 p , the probability of each element being 1 is 1 / p ; S46: Output of the dropout layer Enter with k The output dense layer of neurons and softmax activation function calculates the probability that the input data belongs to each class of labels: ； The dimension of y is k ×N, is the weight matrix, is the bias vector, and the softmax activation function is: ； in, z i is the input vector z The i elements; S47: The probability results of the predicted labels are measured using cross entropy: ； in, Indicates the class to which sample i belongs, synlog indicates the synchronous logarithm, To predict the probability, the training loss is used to update the learnable parameters of the deep neural network through the Adam optimizer until the deep neural network converges.

9. The unsupervised deep learning method based on synthetic data generation according to claim 1, characterized in that: The specific process of step S5 is as follows: The initial unlabeled data sample X Input the deep neural network model pre-trained by synthetic data in step S4, output the corresponding results, and execute downstream tasks.

Images