Multi-source deep adversarial migration method for low-value density data

Abstract

The invention relates to a multi-source deep adversarial migration method for low-value density data, and the method comprises the following steps: carrying out the feature analysis and recognition of the low-value density data, analyzing the features of the low-value density data from the three dimensions of time, attributes and fields, analyzing the inhibition effect of the features on knowledge discovery, and carrying out the recognition of the features of the low-value density data; mining an evolution rule of label data by using a data driving technology, labeling valuable unlabeled samples by using an information entropy theory and an information transfer clustering thought, and evaluating domain similarity by using a Bayesian probability analysis theory; and 4, deep feature adversarial transfer learning of low-value density data: on the basis of the step 1, feature extraction is enhanced through a conditional generative adversarial technology and a deep reserve pool network, and then an adversarial-based deep transfer learning model is constructed. Aiming at the knowledge discovery and prediction application problem of the low-value density data, the multi-source deep anti-migration prediction and application research of the low-value density data is developed, and the method has very remarkable theoretical significance and application value.

Description

Technical Field

[0001] The present invention relates to the field of low-value density data processing, and in particular to a multi-source deep adversarial migration method for low-value density data. Background Art

[0002] In recent years, vast amounts of data have been accumulated across all industries. With the rapid development of next-generation information technology, the processing of this vast amount of data has gradually transitioned from raw statistical analysis to knowledge discovery within massive data. This involves extracting credible, novel, potential, and understandable knowledge from this vast, noisy, random, fuzzy, and incomplete data. However, in reality, the vast majority of this massive data consists of unlabeled samples, resulting in low data value density and hindering the discovery and predictive application of effective knowledge.

[0003] Manually labeled data extraction is costly. For example, in the field of municipal drainage management, experienced engineers are required to accurately judge process data (e.g., determining the monitoring status category of each wastewater treatment unit). Otherwise, the effectiveness of subsequent knowledge utilization will be affected (e.g., misjudgment leading to substandard water quality discharged into receiving water bodies). Traditional machine learning methods are only effective under the general assumption that training and test data come from the same feature space and have the same distribution. When the data distribution or label space changes, new data must be collected and reconstructed, especially when using labeled training data. This is often very costly. For example, water quality prediction for different wastewater treatment processes (wastewater treatment operations have data such as normal, various abnormal, and fault states, with different distributions of information and limited sample sizes for abnormal and fault states) can easily lead to biased prediction results and error shifts. In reality, a large amount of unlabeled data from other auxiliary domains is available. Therefore, reducing the need for re-collecting data and fully utilizing this unlabeled data from auxiliary domains has become a pressing issue for knowledge discovery and prediction applications. Summary of the Invention

[0004] The purpose of the present invention is to provide a multi-source deep adversarial migration method for low-value density data, which can fully utilize low-value density data and reduce the need to re-collect data.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is: a multi-source deep adversarial migration method for low-value density data, comprising the following steps:

[0006] Step 1: Analyze and identify the characteristics of low-value density data from three dimensions: time, attributes, and domain. Analyze its inhibitory effect on knowledge discovery. Use data-driven technology to explore the evolution of labeled data, use information entropy theory and information transfer clustering ideas to annotate valuable unlabeled samples, and use Bayesian probability analysis theory to evaluate domain similarity.

[0007] Step 2: Deep feature adversarial transfer learning of low-value density data. Based on step 1, feature extraction is enhanced through conditional generative adversarial technology and deep reserve pool network, and then a deep transfer learning model based on adversarial is constructed.

[0008] Preferably, the conditional generative adversarial technology is a conditional generative adversarial network, which, like the traditional generative adversarial model, consists of a generator and a discriminator, but the difference is that at the input of the generator, not only the random variable z is input, but also the label information y that the model needs to learn; the network is used to generate data similar to the real data, that is, false data G(z|y). Due to the addition of label information, the prior distribution P(z) generated from the generator is converted into a conditional density P(x|y). That is, this network belongs to supervised learning. When the generated false data enters the discriminator, the input end will also add label information.

[0009] Preferably, the specific steps for data generation using a conditional generative adversarial network are as follows: each time a training sample (label data) is fed into the conditional generative network for iteration, corresponding false data will be generated by the generator each time; then, each type of generated data will be evaluated for accuracy through a predictor, and the model will only save the generator with better prediction accuracy than the last time (i.e., the optimal generator) during multiple iterations; finally, data generation is performed through this optimal generator.

[0010] Preferably, the deep reservoir network enhanced feature extraction combines the deep learning framework and the reservoir computing technology to construct a deep reservoir network structure, which includes K reservoirs and (K-1) encoders. For the i-th reservoir, the update equation is similar to the general echo state network, and the corresponding echo state representation can be obtained; then the reservoir state of the previous layer is encoded using unsupervised learning to obtain the feature representation of the previous reservoir; finally, the features of each level obtained before are integrated, and the implementation path is: the deep reservoir network splices the original input signal, the echo state generated by the middle reservoir layer, and the echo state of the last layer through direct connection, feature connection and output weight respectively.

[0011] Preferably, the adversarial deep transfer learning method combines the conditional generative adversarial network and the deep reservoir network to enhance feature extraction, constructs a domain-shared feature extraction network, and thus finds transferable representations suitable for the source domain and the target domain, that is, the domain adaptation problem; the domain adaptation problem eliminates the process of generating samples, and uses the generative adversarial network to regard the target domain data as generated samples. Therefore, the purpose of the generator has changed. It is no longer to generate samples, but to extract features, that is, to extract shared features from the source domain and the target domain, so that the discriminator cannot distinguish whether the extracted features are from the source domain or the target domain.

[0012] Preferably, step one includes the following specific steps:

[0013] ①Analyze the characteristics of labeled samples, reveal the evolution mechanism of labeled samples, and reconstruct the labeled sample set;

[0014] ②Analyze the value of unlabeled samples, analyze the unlabeled samples that do not contribute, and label the most information-rich unlabeled samples;

[0015] ③ Domain transfer sample identification: Based on the Bayesian probability analysis theory, the similarity between each source task and the target task is evaluated, and the most appropriate source task is selected from the source task space for transfer;

[0016] Assume that the space containing M source tasks and 1 target task is (S,T)=(S1,S2,…,S M ,T), then the likelihood of each source task and target task is

[0017]

[0018] The likelihood X between the source task and the target task k for

[0019]

[0020] Where, P(S k ) is Model S k The prior probability, χ u Represents the available u-th target sample and source task S k The total matching degree of the source task S can be obtained by normalizing the likelihood of each source task k The task similarity with T; sort by descending order of sample matching probability, and select the top ranked samples as the migration sample set.

[0021] Preferably, the label sample feature analysis is based on the data-driven modeling idea, and time-frequency-modal analysis is performed on the label samples in the time dimension to improve the information representation; low-dimensional feature analysis is performed on the label samples in the attribute dimension to extract the low-dimensional features that best reflect the process characteristics, reveal the evolution mechanism of the label samples, and reconstruct the label sample set.

[0022] Preferably, information entropy technology is used to preliminarily select the undetermined label data, that is, the entropy value x * The larger the entropy value, the greater the uncertainty of the system, and vice versa. Therefore, non-labeled sample data with large entropy value can be selected as the undetermined labeled data;

[0023] x * =argmax x [-∑P θ (y i |x)·lnP θ(y i |x)]

[0024] Where θ represents a set of trained machine learning model parameters. The unlabeled sample y is the category with the highest model prediction probability for the labeled target x, and its probability distribution is represented by P(.).

[0025] Then, based on the clustering idea of ​​"information transmission" between data points, the reconstructed label samples are used as the benchmark, and the attractor propagation technology is used to screen the unlabeled data (unlabeled data set); it mainly includes two iterative processes, attracting information and attributing information, namely

[0026]

[0027]

[0028] In the formula, the attraction information r(i,j) indicates the degree to which data j (labeled sample) is suitable as the cluster center of data i (undetermined labeled data); the attribution information a(i,j) indicates the degree to which data i (undetermined labeled data) selects data j (labeled sample) as its cluster center; if the information result remains unchanged within the set number of iterations, the undetermined labeled data is retained, otherwise it is deleted, and then a non-labeled sample selection mechanism and approach is established to expand the sample set.

[0029] The technical effects of the present invention are:

[0030] This invention aims at the problem of knowledge discovery and prediction application of low-value density data. By studying the characteristics of labeled samples and auxiliary domain non-label samples, combining deep intelligent technologies such as adversarial generative models, deep learning, and migration prediction, and targeting the system predictive management needs, it conducts multi-source deep adversarial migration prediction and application research on low-value density data, which has very significant theoretical significance and application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A roadmap for low-value-density data feature analysis and identification.

[0032] Figure 2 Roadmap for deep feature adversarial transfer learning solutions for low value density data.

[0033] Figure 3 Flowchart of the migration prediction model. DETAILED DESCRIPTION

[0034] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the following embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0035] The specific technical solutions of the present invention are as follows:

[0036] (1) Low value density data feature analysis and identification scheme (such as Figure 1 shown)

[0037] We analyze the characteristics of low-value density data from three dimensions: time, attributes, and domains, analyze its inhibitory effect on knowledge discovery, use data-driven technology to explore the evolution of labeled data, use information entropy theory and information transfer clustering ideas to annotate valuable unlabeled samples, and use Bayesian probability analysis theory to evaluate domain similarity. Specifically,

[0038] ① Label sample feature analysis.

[0039] Any application scenario is a complex open system. Influenced by the coupling of multiple variables, the data fluctuations of the target are diverse. This is manifested in the following ways: in the time dimension, such regularities are difficult to fully understand under the scale-invariant rule, especially for data fluctuations in non-stable states; in the attribute dimension, high-dimensional variables, while containing rich process information, also contain a lot of redundant information, which interferes with data modeling. Therefore, based on data-driven modeling, time-frequency-modal analysis (such as variable mode decomposition) is performed on labeled samples in the time dimension to improve information representation; in the attribute dimension, low-dimensional feature analysis (such as nonlinear manifold technology) is performed on labeled samples to extract low-dimensional features that best reflect process characteristics. Through this research, the evolution mechanism of labeled samples is revealed and the labeled sample set is reconstructed.

[0040] ②Analysis of the value of unlabeled samples.

[0041] The information content of unlabeled samples cannot be ignored (they account for a large proportion), but not all unlabeled samples contribute to modeling, so it is not necessary to label all unlabeled samples. The general idea is that most of the unlabeled samples rich in information will be labeled instead of labeling all unlabeled samples. Therefore, based on the uncertainty theory (samples with high uncertainty contain rich information), the information entropy technology is used to preliminarily select the unlabeled data. That is, the entropy value x * The larger the entropy value, the greater the uncertainty of the system (the higher the value of the data information), and vice versa. Therefore, non-labeled sample data with large entropy values ​​can be selected as undetermined labeled data.

[0042] x *=argmax x [-∑P θ (y i |x)·lnP θ (y i |x)]

[0043] Where θ represents a set of trained machine learning model parameters. The unlabeled sample y is the class with the highest model prediction probability for the labeled target x, and its probability distribution is represented by P(.).

[0044] Then, based on the clustering idea of ​​"information transmission" between data points, the reconstructed label samples (research ①) are used as the benchmark, and the attractor propagation technology is used to screen the unlabeled data (unlabeled data set). It mainly includes two iterative processes, attracting information and attributing information, namely

[0045]

[0046] In this formula, the attraction information r(i, j) indicates the degree to which data j (labeled sample) is suitable as the cluster center for data i (unlabeled data); the attribution information a(i, j) indicates the degree to which data i (unlabeled data) chooses data j (labeled sample) as its cluster center. If the information remains unchanged within a set number of iterations, the unlabeled data is retained; otherwise, it is deleted. Through this research, a mechanism and approach for selecting unlabeled samples is established, expanding the sample set.

[0047] ③ Identification of domain migration samples.

[0048] In reality, multi-source domain sample sets exhibit internal variations (e.g., fluctuations in normal / abnormal data), as well as differences between source and target domains. Considering only the single source domain with the greatest similarity to the target domain as transfer information results in information loss. Considering all source domains as transfer information leads to biased modeling, particularly under-learning of features from rare domains (e.g., abnormal fluctuations). Therefore, Bayesian probability analysis is used to evaluate the similarity between each source task and the target task, and the most appropriate source task is selected from the source task space for transfer.

[0049] Assume that the space containing M source tasks and 1 target task is (S,T) = (S1,S2,…,SM,T), then the likelihood of each source task and target task is

[0050]

[0051] The likelihood X between the source task and the target task k for

[0052]

[0053] Where, P(S k ) is Model S k The prior probability, χ u Represents the available u-th target sample and source task S k By normalizing the likelihood of each source task, we can obtain the source task S k The task similarity with T. Arrange the samples in descending order according to the sample matching probability, and select the top ranked samples (the number is determined by the set rules) as the migration sample set.

[0054] (2) Deep feature adversarial transfer learning scheme for low value density data

[0055] Features are information extracted from data that is useful for predicting results. Therefore, based on the reconstruction of the sample space (identification of labeled samples and unlabeled samples), we study conditional generative adversarial techniques, deep reservoir networks, and other methods to enhance feature extraction, and then build a deep transfer learning model based on adversarial learning. Specifically,

[0056] ① Research on conditional generative adversarial networks.

[0057] Model accuracy depends not only on the "quality" of data, but also on the "quantity". In a low-value density data environment, reconstructed samples (labeled and unlabeled sample identification) still face the problem of insufficient model training, and there is also the problem of sample imbalance within / between domains. Adversarial learning, as a "production machine", can generate some sample information with high precision, provide sufficient training samples for building deep learning models and ensure the balance of samples in various fields. Conditional Generative Adversarial Network (GAN) Figure 2 ), like the traditional generative adversarial model, consists of a generator and a discriminator, but the difference is that at the input of the generator, not only the random variable z is input, but also the label information y that the model needs to learn. The generative network is used to generate data similar to the real data, that is, fake data G(z|y). Due to the addition of label information, the probability distribution generated by the prior distribution P(z) from the generator becomes the conditional density P(x|y), that is, this network belongs to supervised learning. When the generated fake data enters the discriminator, the input end will also add label information. The purpose of this is to better learn the current label information. Therefore, the conditional generative adversarial network not only inherits the advantage of not requiring data distribution assumptions, but also adds conditional constraints to improve the quality of data generation (guideable generated data).

[0058] During model training, the following may occur: 1) mode collapse, where the model can only learn part of the data distribution; 2) the training loss curve is in an oscillating state; 3) the gradient disappears. These problems will make it impossible to obtain an optimal generator. Based on previous research, a relatively robust idea is proposed, that is, the generator saved is a generator with better prediction accuracy than the previous one. First, each time the training sample (label data) is fed into the conditional generative network for iteration, the corresponding fake data will be generated by the generator each time; then, each type of generated data will be evaluated for accuracy through a predictor, and the model will only save the generator with better prediction accuracy than the last time (that is, the optimal generator) during multiple iterations. Finally, data is generated through this optimal generator.

[0059] ② Research on deep reserve pool network.

[0060] In the era of big data, deep technology has been widely used in the field of feature learning. Reservoir computing (a deformation of recurrent neural networks) has the advantages of fast convergence speed, suitability for dynamic data modeling, and no gradient disappearance problem. Many previous works have shown that it has advantages in feature extraction and learning. However, a single-layer reservoir cannot effectively extract features from complex input structure data, especially noisy data (such as incorrect labeling of non-labeled data), and its recognition effect cannot meet the requirements of prediction accuracy. In addition, the regression decoding process from the activation signal to the output signal on the single-layer reservoir neuron is relatively simple, and the advantages of the reservoir for time series dynamic prediction modeling have not been fully utilized. Therefore, combining the deep learning framework and reservoir computing technology, a deep reservoir network structure ( Figure 2 ).

[0061] This structure consists of K reservoirs and (K-1) encoders. For the i-th reservoir, an update equation similar to that of a general echo state network is used to obtain the corresponding echo state representation. This is then encoded using unsupervised learning of the reservoir state of the previous layer to obtain a feature representation of the previous reservoir. Finally, the features obtained from each layer are integrated. The implementation path is as follows: the deep reservoir network concatenates the original input signal, the echo state generated by the intermediate reservoir layer, and the echo state of the final layer through direct connections, feature connections, and output weights.

[0062] ③ Research on deep transfer learning methods based on adversarial learning.

[0063] Due to the complexity and evolution of real-world systems, it is impossible to collect all information. Therefore, when modeling new tasks (target domains), the purpose of this research is to make full use of source domain samples (labeled and unlabeled sample identification) and improve the utilization rate of low-value data. Figure 2), drawing on the adversarial learning strategy from study ① and the deep feature learning technology from study ②, constructs a domain-shared feature extraction network to find transferable representations applicable to both the source and target domains, i.e., domain adaptation. This is the foundation for effective transfer.

[0064] Domain adaptation eliminates the sample generation process and instead uses a generative adversarial network to treat the target domain data as generated samples. Therefore, the purpose of the generator changes from sample generation to feature extraction: extracting shared features from the source and target domains, making it impossible for the discriminator to distinguish between the extracted features and the source domain. The network loss for domain adaptation consists of two parts: training loss (label predictor loss) and domain discriminator loss. The parameters of the label predictor are updated by minimizing the objective function, while the parameters of the domain discriminator are updated by maximizing the objective function.

[0065] The core task of domain adaptation is to reduce the distribution discrepancy between the source and target domains, thereby enabling the source domain model to better adapt to the target domain data. To ensure effective transfer, accurately measuring and reducing the feature distribution discrepancy between the source and target domains is crucial. To this end, this paper introduces Multi-Kernel Maximum Mean Discrepancy (MK-MMD) as a tool for measuring inter-domain feature distribution discrepancy. However, relying solely on MK-MMD to measure distribution discrepancy can lead the model to over-focus on aligning feature distributions while neglecting the accuracy of the prediction task itself. To further improve the performance of the domain adaptation module, this paper proposes a new transfer loss function strategy that combines the MK-MMD loss (LossMK-MMD) with the regression loss (LossPred). These two losses are weighted together to optimize the training process of the target model.

[0066] In addition, this paper designs a dynamic adjustment mechanism for the transfer loss function, which can adaptively adjust the weights of the distribution difference loss and regression loss according to different stages of the training process, so as to achieve the optimal transfer effect; specifically, this mechanism can ensure that the model quickly grasps the basic laws of the target task in the early stage of training, while avoiding premature or excessive emphasis on the distribution alignment problem between the source domain and the target domain; as the training deepens, the model gradually introduces more distribution alignment constraints to further optimize the transfer of source domain knowledge, and ultimately achieve effective transfer of source domain knowledge, while optimizing the prediction performance of the target task, thereby obtaining better transfer learning effects.

[0067] Distribution difference loss

[0068] The distribution difference loss is primarily used to measure the distributional discrepancy between the source and target domains. By minimizing this loss, it ensures that the features of the source and target domains are as similar as possible in terms of distribution. The MMD method maps the source and target domains into the same Reproducing Kernel Hilbert Space (RHKS) and measures the inter-domain difference by comparing the means of the two probability distributions in the RHKS. The kernel function for MMD can be a single Gaussian kernel function or a linear kernel function. However, different kernel functions may capture different data characteristics, making it difficult to guarantee the optimal choice of a single kernel function. Selecting an appropriate kernel function is particularly challenging when the data distribution is complex. To address this issue, this paper employs the MK-MMD method, which constructs an optimal kernel function by performing a weighted combination of multiple kernel functions and finding the optimal weights. This method not only fully leverages the advantages of multiple kernel functions but also automatically adapts to data characteristics by optimizing the weights, thereby improving the accuracy and robustness of inter-domain similarity assessment.

[0069] Distribution Difference Loss MK-MMD Defined as:

[0070]

[0071] Where: X={x1,x2,…,x n} is the source domain input sequence, n is the length of the source domain input sequence; Y = {y1, y2, ..., y m} is the target domain input sequence, m is the input sequence length; ||.|| H represents the RKHS space; φ(.) is the mapping function.

[0072] After expanding the above formula, the inner product in RKHS can be converted into a kernel function, expressed as:

[0073]

[0074] In order to construct the optimal kernel function K(.), it is necessary to perform weighted summation of multiple kernel functions, which can be expressed as:

[0075]

[0076] Where: β represents weight, k represents kernel function, q is the number of kernel functions, β μ ≥0 is to ensure that the combined core is still a valid core.

[0077] Regression Loss

[0078] Regression loss is mainly used to optimize the prediction performance of the target task and ensure that the target task is learned accurately during the training process.

[0079] Regression Loss Pred Defined as:

[0080] Loss pred =J(P(y1,y2,...,y m ),y t )

[0081] Where: y t is the t-th target value; J(.) is the mean square error function; P(.) is the regression predictor, that is, the Transformer decoder in the TimesNet model selected in this study.

[0082] Dynamic migration loss function

[0083] Dynamic migration loss function Loss Transfer The structure is:

[0084] Loss Transfer =Loss pred +λLoss MK-MMD

[0085] Among them, λ is the dynamic weight of the distribution difference loss, which is used to control the importance of the distribution difference loss in the migration loss and is defined as:

[0086]

[0087] α represents the current training progress, that is, the ratio of the current training epoch number to the total training epoch number.

[0088] During training, the optimization of dynamic transfer loss follows these steps: 1) Optimizing regression loss. In the early stages of training, the target task model is still in its early stages of learning. At this point, feature alignment may interfere with the optimization of the target task. Therefore, the weight of the distribution difference loss is set low at this stage, aiming to enable the model to quickly grasp the basic laws of the target task and minimize the impact of feature alignment, thereby accelerating the learning of the target task. 2) Balancing distribution difference loss and regression loss. In the middle stages of training, the model has initially grasped the basic laws of the target task. At this point, the feature difference between the source and target domains begins to narrow. To further optimize the transfer of source domain knowledge, the model will gradually enter more distribution alignment constraints. In this stage, the model not only continues to optimize the prediction ability of the target task, but also strengthens the alignment of the distribution difference between the source and target domains. 3) Weighted distribution difference loss. In the later stages of training, the learning of the target task has made some progress. In this stage, the model will focus more on the transfer of source domain knowledge, making the distribution difference loss dominant to further align the feature distribution without affecting the optimization of the target task.

[0089] Migration Prediction Framework

[0090] In order to achieve effective transfer learning from the source domain to the target domain, a systematic training process is proposed, which aims to improve the prediction accuracy of the target domain task through steps such as pre-training, fine-tuning and optimizing the loss function. The modeling process of the model is as follows Figure 3 The specific steps can be described as follows:

[0091] Step 1: Data preprocessing;

[0092] The source domain and target domain data are standardized, and a sliding window is used to cut and sample the data.

[0093] Step 2: Model pre-training;

[0094] (1) Use all source domain data to train the model and save the source domain model parameters and training results;

[0095] (2) Design the fine-tuning ratio to learn the specific features of the target domain; the fine-tuning ratio is set to 80% at the highest (i.e., 80% of the samples in the target domain participate in model fine-tuning), and the fine-tuning ratio is set to 10% at the lowest, with a ratio interval of 10%; save the fine-tuned model parameters and fine-tuning results.

[0096] Step 3: Domain transfer training;

[0097] (1) The source domain / target domain feature samples generated by the source domain / target domain feature extraction network (the results of the second step pre-training and fine-tuning process) are used as the input of the distribution difference loss metric to calculate the distribution difference loss LossMK-MMD; the prediction results output by the model pre-training (obtained by the second step fine-tuning) are used to calculate the regression loss LossPred;

[0098] (2) Back-propagate the transfer loss function LossTransfer, iteratively minimize the transfer loss (termination condition is the maximum number of iterations or reaching the error target), and optimize the model.

[0099] Step 4: Migration model generation and prediction.

[0100] When the migration loss function is minimized, the migration model is obtained; using this model, the test task is carried out on the target domain samples.

[0101] It should be noted that, in this article, the terms "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that includes a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements that are inherent to such process, method, article or apparatus.

[0102] This article uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only used to help understand the method of the present invention and its core ideas. The above is only a preferred implementation method of the present invention. It should be pointed out that due to the limitations of textual expression, there are objectively infinite specific structures. For ordinary technicians in this technical field, without departing from the principles of the present invention, they can make several improvements, modifications or changes, and can also combine the above technical features in an appropriate manner; these improvements, modifications, changes or combinations, or the direct application of the inventive concept and technical solution to other occasions without improvement, should be regarded as the scope of protection of the present invention.

Claims

1. A multi-source deep adversarial migration method for low-value density data, characterized by: The steps include: Step 1: Analyze and identify the characteristics of low-value density data from three dimensions: time, attributes, and domain. Analyze its inhibitory effect on knowledge discovery. Use data-driven technology to explore the evolution of labeled data, use information entropy theory and information transfer clustering ideas to annotate valuable unlabeled samples, and use Bayesian probability analysis theory to evaluate domain similarity. Step 2: Deep feature adversarial transfer learning of low-value density data. Based on step 1, feature extraction is enhanced through conditional generative adversarial technology and deep reserve pool network, and then a deep transfer learning model based on adversarial is constructed.

2. The multi-source deep adversarial migration method for low-value density data according to claim 1 is characterized in that: Conditional generative adversarial technology is a conditional generative adversarial network, which, like the traditional generative adversarial model, consists of a generator and a discriminator. The difference is that at the input of the generator, not only the random variable z is input, but also the label information y that the model needs to learn; the network is used to generate data similar to the real data, that is, false data G(z|y). Due to the addition of label information, the prior distribution P(z) generated from the generator is converted into the conditional density P(x|y). That is, this network belongs to supervised learning. When the generated false data enters the discriminator, the input will also add label information.

3. The multi-source deep adversarial migration method for low-value density data according to claim 2 is characterized in that: The specific steps for data generation using conditional generative adversarial networks are as follows: each time a training sample (labeled data) is fed into the conditional generative network for iteration, corresponding fake data is generated by the generator each time; then, the accuracy of each type of generated data is evaluated by a predictor, and the model only saves the generator with better prediction accuracy than the last time (i.e., the optimal generator) during multiple iterations; finally, data is generated through this optimal generator.

4. The multi-source deep adversarial migration method for low-value density data according to claim 3 is characterized in that: Deep reservoir network enhanced feature extraction combines the deep learning framework and reservoir computing technology to construct a deep reservoir network structure, which contains K reservoirs and (K-1) encoders. For the i-th reservoir, the update equation is similar to the general echo state network to obtain the corresponding echo state representation; then the reservoir state of the previous layer is encoded using unsupervised learning to obtain the feature representation of the previous reservoir; finally, the features of each level obtained before are integrated. The implementation path is: the deep reservoir network splices the original input signal, the echo state generated by the middle reservoir layer, and the echo state of the last layer through direct connection, feature connection and output weight respectively.

5. The multi-source deep adversarial migration method for low-value density data according to claim 4 is characterized in that: The adversarial deep transfer learning method combines the conditional generative adversarial network and the deep reservoir network to enhance feature extraction, constructing a domain-shared feature extraction network to find transferable representations suitable for the source and target domains, which is the domain adaptation problem. The domain adaptation problem eliminates the process of generating samples and uses the generative adversarial network to treat the target domain data as generated samples. Therefore, the purpose of the generator has changed. It is no longer to generate samples, but to extract features, that is, to extract shared features from the source and target domains, so that the discriminator cannot distinguish whether the extracted features are from the source or target domain.

6. The multi-source deep adversarial migration method for low-value density data according to claim 1 is characterized in that: Step 1 includes the following specific steps: ①Analyze the characteristics of labeled samples, reveal the evolution mechanism of labeled samples, and reconstruct the labeled sample set; ②Analyze the value of unlabeled samples, analyze the unlabeled samples that do not contribute, and label the most information-rich unlabeled samples; ③ Domain transfer sample identification: Based on the Bayesian probability analysis theory, the similarity between each source task and the target task is evaluated, and the most appropriate source task is selected from the source task space for transfer; Assume that the space containing M source tasks and 1 target task is (S,T)=(S1,S2,…,S M ,T), then the likelihood of each source task and target task is The likelihood X between the source task and the target task k for Where, P(S k ) is Model S k The prior probability, χ u Represents the available u-th target sample and source task S k The total matching degree of the source task S can be obtained by normalizing the likelihood of each source task k The task similarity with T; sort by descending order of sample matching probability, and select the top ranked samples as the migration sample set.

7. The multi-source deep adversarial migration method for low-value density data according to claim 6 is characterized in that: Label sample feature analysis, based on data-driven modeling ideas, conducts time-frequency-modal analysis on label samples in the time dimension to improve information representation; conducts low-dimensional feature analysis on label samples in the attribute dimension to extract low-dimensional features that best reflect process characteristics, reveal the evolution mechanism of label samples, and reconstruct the label sample set.

8. The multi-source deep adversarial migration method for low-value density data according to claim 7 is characterized in that: Use information entropy technology to preliminarily select undetermined label data, that is, entropy value x * The larger the entropy value, the greater the uncertainty of the system, and vice versa. Therefore, non-labeled sample data with large entropy value can be selected as the undetermined labeled data; x * =argmax x [-∑P θ (y i |x)·lnP θ (y i |x)] Where θ represents a set of trained machine learning model parameters. The unlabeled sample y is the category with the highest model prediction probability for the labeled target x, and its probability distribution is represented by P(.). Then, based on the clustering idea of ​​"information transmission" between data points, the reconstructed label samples are used as the benchmark, and the attractor propagation technology is used to screen the unlabeled data (unlabeled data set); it mainly includes two iterative processes, attracting information and attributing information, namely In the formula, the attraction information r(i,j) indicates the degree to which data j (labeled sample) is suitable as the cluster center of data i (undetermined labeled data); the attribution information a(i,j) indicates the degree to which data i (undetermined labeled data) selects data j (labeled sample) as its cluster center; if the information result remains unchanged within the set number of iterations, the undetermined labeled data is retained, otherwise it is deleted, and then a non-labeled sample selection mechanism and approach is established to expand the sample set.

Images