Industrial big data driven longitudinal federal migration anomaly detection method and system
Through the vertical federal migration anomaly detection method, vertical federal feature extraction, domain adaptation and joint domain alignment modules are used to solve the problems of data privacy and accuracy in transfer learning in industrial big data, and realize the industry with large sample scarcity and large distribution differences in the sample Efficient abnormality detection in scenarios.
Patent Information
- Application Number
- CN202411034010.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-30
- Publication Date
- 2024-11-12
Smart Images

Figure CN118940003A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of industrial intelligence technology, and specifically relates to a method and system for detecting anomalies in vertical federated migration driven by industrial big data. Background Art
[0002] Industrial big data is a key production factor. It involves data from the entire life cycle of products and services in the industrial field, including data generated and used in R&D design, production and manufacturing, business management, operation and maintenance services, and data in the industrial Internet platform. The collection method of industrial big data is mainly based on sensors, supplemented by manual uploading, covering all aspects of industrial production. Industrial big data also has complex dynamic system characteristics, including 4V characteristics such as large scale, fast speed, diverse types, and low quality, as well as new characteristics such as multimodality, strong correlation, and high throughput. These characteristics make industrial big data face a series of technical challenges in analysis and application. In order to obtain more knowledge from these massive data to improve competitiveness, machine learning has become an important technology in the industrial field.
[0003] In the analysis of industrial big data, data anomaly detection and anomaly warning are very important links. By analyzing and mining industrial data, we can timely detect abnormal status of production equipment, abnormal fluctuations in sales process and abnormal behavior in operation and maintenance process, and prevent sudden abnormalities in key links, so as to take corresponding measures to ensure the normal operation of the entire industrial life cycle, thereby meeting the development requirements of high efficiency, high profit and automation of the industrial industry. Data-driven machine learning methods have been widely used in the field of industrial anomaly detection. The goal is to extract multiple features from sensor signals of industrial big data with diverse modes and changing working conditions (such as current, voltage, vibration, temperature and acoustic emission, etc.) to effectively model and distinguish abnormal modes from normal modes. The main methods rely on a large number of well-labeled samples to train various machine learning (ML) models (e.g., SVM, DNN), which are significantly better than traditional anomaly detection methods based on partial system mechanisms or expert experience.
[0004] However, in actual industrial scenarios, due to the extreme scarcity of abnormal samples owned by a large number of industrial participants, traditional data-driven methods based on a large number of labeled samples cannot be used. For example, for a factory with newly deployed equipment, equipment abnormalities are usually low-probability events, so the data abnormality rate is generally low, and it usually takes a long time to collect samples with certain "abnormal" labels when anomalies occur. How to build an effective anomaly detection model with a small number or even zero abnormal labels has become a bottleneck problem facing the current industry. To solve this problem, under the assumption of independent and identical distribution (IID), the anomaly detection model can be trained with more labeled related samples on other similar industrial participants (i.e., source domains), and then the trained model can be deployed on participants with scarce samples (i.e., target domains). However, the IID assumption is almost impossible to guarantee in real scenarios. Due to multiple differences in monitoring settings, workloads, transmission paths, noise interference, abnormality levels, etc., there must be obvious distribution differences in samples of participants under different working conditions, that is, different participants usually have different sample domains. Transfer learning can be used to build and migrate models in different fields and has been widely used in industrial anomaly detection.
[0005] However, existing migration methods require exchanging samples between the source domain and the target domain to extract common domain invariant information. However, considering the impact of stringent regulations such as the General Data Protection Regulation (GDPR) and other related factors, the current anomaly detection methods based on transfer learning cannot effectively guarantee the privacy of industrial data participants, and the accuracy of anomaly label detection still needs to be further improved. Summary of the invention
[0006] The purpose of the present invention is to provide an industrial big data driven vertical federated migration anomaly detection method and system to solve the problem that the current anomaly detection method based on transfer learning cannot effectively guarantee the privacy of industrial data participants, and the accuracy of anomaly label detection still needs to be further improved.
[0007] In order to achieve the above object, the present invention adopts the following technical scheme:
[0008] The present invention provides an industrial big data driven vertical federation migration anomaly detection method, comprising the following steps:
[0009] S1. Obtain a source domain dataset and a target domain dataset from an industrial scenario; wherein the source domain dataset is constructed based on industrial data with known abnormal labels, and the target domain dataset is constructed based on industrial data without abnormal labels;
[0010] S2. Based on the preset vertical federation migration model:
[0011] Perform vertical federated feature extraction: map the source domain dataset and the target domain dataset into a common feature space to obtain latent features; perform domain adaptation: extract domain-invariant and distinguishable features from latent features; perform joint domain alignment: align the distance between domains, map the domain-invariant and distinguishable features to obtain the final anomaly label;
[0012] Among them, the preset vertical federation migration model is constructed through the vertical federation feature extraction module, the domain adaptation module and the joint domain alignment module, and then obtained through training;
[0013] Among them, the training is carried out based on the unsupervised longitudinal federated migration model pre-training process and the longitudinal federated migration model training process.
[0014] In some embodiments, in S2, the vertical federated feature extraction is based on a convolutional neural network and a spatial pyramid method to map the source domain dataset and the target domain dataset into a common feature space.
[0015] Furthermore, in S2, the specific operation of vertical federated feature extraction is:
[0016] By adding a spatial pyramid pooling layer before the fully connected layer of the convolutional neural network, the spatial pyramid pooling layer includes adaptive convolution layers of different levels. The adaptive convolution automatically adjusts the shape of the convolution kernel, padding, and stride parameters by fixing the size of the output feature map. The shape and size of the convolution kernel, padding, and stride are inferred by the following formulas (1), (2), and (3). The adaptive convolution layer forms potential features.
[0017]
[0018] kernel=H in -(H out -1)*stride(2);
[0019] padding=0 (3);
[0020] Among them, stride represents the stride, padding represents the padding, and kernel represents the convolution kernel. Indicates rounding down, H in Indicates the size of the input data, H out Indicates the feature output size.
[0021] In some embodiments, in S2, domain adaptation is based on the principle of conditional domain adversarial learning to reduce the difference in feature distributions of different domains, and extract features with domain invariance and distinguishability from latent features.
[0022] Furthermore, in S2, the specific operations of domain adaptation are:
[0023] The latent feature alignment loss and the abnormal label classification loss of the source domain dataset are optimized based on the adversarial learning principle; on the one hand, the objective function of the latent feature alignment loss is as follows:
[0024]
[0025] Among them, E represents the expectation, f S and g S They represent the potential features of the source domain and the classifier results output by the potential features, respectively. T and g T Corresponding to the latent features in the target domain and the classifier results output by the latent features; represents a multilinear map;
[0026] The sample weight method of formula (5) below is used to avoid negative samples from affecting training:
[0027]
[0028] Among them, p represents the final prediction probability of the classifier for each category;
[0029] The optimization method based on joint domain adversarial learning is used to optimize the latent feature alignment loss; by minimizing the objective function of the latent feature alignment loss, the parameters of the feature extractor F and the classifier C are optimized, while maximizing the objective function of the latent feature alignment loss to optimize the parameters of the domain discriminator D. The optimization process of each model parameter is as follows:
[0030]
[0031] Among them, t represents the tth iteration update, and They represent the parameters of the feature extractor F, classifier C, and domain discriminator D of the t-th training respectively;
[0032] On the other hand, the objective function of the abnormal label classification loss of the source domain dataset is as follows:
[0033]
[0034] in, and Represents the i-th data sample and the corresponding label in the source domain respectively; N S The source domain D S The total number of samples; F S Represents the feature extractor on the source domain.
[0035] In some embodiments, in S2, joint domain alignment is a difference-based label distribution distance alignment method, which maps features with domain invariance and distinguishability into final anomaly labels to achieve anomaly detection in the target domain without anomaly labels.
[0036] Furthermore, in S2, the specific operation of joint domain alignment is:
[0037] By adding the difference-based label distribution distance alignment method as a joint domain alignment to calculate the output label alignment loss, the objective function of the joint domain label alignment loss is as follows:
[0038]
[0039] in, The latent features representing the source domain input in the classification network are represented as the output of the kth layer, It means that each layer output of the classifier is projected into a Hilbert space through a multidimensional linear mapping, |L| represents the number of layers in the classifier, and the last two layers of the classifier output are selected for alignment of anomaly detection tasks on different domains.
[0040] In some implementations, in S2, the model pre-training process is performed in an unsupervised longitudinal federated migration to reduce the training time and speed up the training convergence speed; specifically:
[0041] The model input is the source domain dataset, the source domain feature extractor and classifier parameters are randomly initialized, and then the objective function of the pre-training loss of the longitudinal federated migration model is calculated as follows:
[0042]
[0043] Then back propagation updates the source domain feature extractor and classifier parameters as follows:
[0044]
[0045] Where μ is the learning rate; Represents the gradient information of the loss objective function.
[0046] In some implementations, in S2, the vertical federation migration model training process trains the vertical federation migration model based on a local and central server joint training method, specifically:
[0047] The model input is the source domain dataset and the target domain dataset. The model parameters are initialized to the model parameters obtained after pre-training the longitudinal federated migration model. The source domain dataset is input into the feature extractor and classifier to obtain the potential features and abnormal label classification loss. The potential features and gradient information are sent to the central server. The central server is waited for to return the update information. The feature extractor and classifier parameters are updated by the following formula:
[0048]
[0049] Among them, λ and β are hyperparameters; f S =F S (X S ) represents the latent features extracted from the source domain;
[0050] Randomly select target domain samples in the target domain and obtain the latent features f after the feature extractor T And the objective function, send the potential features to the central server, block waiting for parameter updates from the central server, and update the feature extractor and classifier parameters through the following formula:
[0051]
[0052] Among them, μ represents the learning rate, and t represents the tth iteration update;
[0053] The central server receives all uploaded potential features and the corresponding labels of the source domain, performs reverse labeling, and calculates the objective function as follows:
[0054]
[0055] in, Represents the abnormal label classification loss of the source domain dataset; represents latent feature alignment loss; stands for joint domain label alignment loss;
[0056] The central server forward propagates to calculate the loss, backpropagates to obtain the gradient, and transmits the gradient information to the source domain and the target domain.
[0057] The present invention also provides an industrial big data driven vertical federation migration anomaly detection system, comprising a vertical federation feature extraction module, a domain adaptation module and a joint domain alignment module; wherein:
[0058] A vertical federated feature extraction module is used to map the source domain dataset and the target domain dataset into a common feature space to obtain latent features;
[0059] The domain adaptation module is used to extract domain-invariant and distinguishable features from latent features;
[0060] The joint domain alignment module is used to align the distances between domains and map the features with domain invariance and distinguishability to obtain the final anomaly labels.
[0061] Compared with the prior art, the industrial big data-driven vertical federated migration anomaly detection method and system of the present invention have the following beneficial technical effects.
[0062] The present invention discloses an industrial big data driven vertical federated migration anomaly detection method, comprising the following steps: S1, obtaining a source domain data set and a target domain data set from an industrial scenario; wherein the source domain data set is constructed based on industrial data with known anomaly labels, and the target domain data set is constructed based on industrial data without anomaly labels; S2, based on a preset vertical federated migration model: performing vertical federated feature extraction: mapping the source domain data set and the target domain data set to a common feature space to obtain potential features; performing domain adaptation: extracting features with domain invariance and distinguishability from the potential features; performing joint domain alignment: aligning the distance between domains, mapping the features with domain invariance and distinguishability to obtain the final anomaly label; wherein the preset vertical federated migration model is constructed by a vertical federated feature extraction module, a domain adaptation module, and a joint domain alignment module, and then obtained by training; wherein the training is based on an unsupervised vertical federated migration model pre-training process and a vertical federated migration model training process. Based on the above, the present invention uses an unsupervised vertical federated migration method to help participants with missing anomaly labels to build an anomaly detection model under the premise of protecting data privacy and security in an actual industrial scenario where data is heterogeneous and anomaly labels are missing. The present invention can realize federated transfer learning across participants based on the federated learning architecture, so that all original data can be retained locally by the participants, thereby avoiding data leakage. The present invention maps heterogeneous features on different domains into a common feature space by constructing a vertical federated feature extraction module; constructs a domain adaptation module to reduce the difference in feature distribution on different domains, and further extracts features with domain invariance and distinguishability; constructs a joint domain alignment module to align the distance between domains to achieve anomaly detection in the target domain without abnormal labels; unsupervised federated model training process, no need for participants to share data, speeds up model training convergence and prevents model overfitting. The present invention has high accuracy and strong generalization, and can meet the application requirements of anomaly detection in actual industrial processes. BRIEF DESCRIPTION OF THE DRAWINGS
[0063] The drawings in the specification are used to provide further understanding of the present invention and constitute a part of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.
[0064] Figure 1A schematic diagram of a model of an industrial big data driven vertical federated migration anomaly detection method and system applied to an industrial anomaly detection scenario in the present invention;
[0065] Figure 2 A schematic diagram of the basic architecture of an unsupervised vertical federated migration anomaly detection model in a method for detecting anomalies of vertical federated migration driven by industrial big data and a system of the present invention;
[0066] Figure 3 A schematic diagram of a basic training process of a method for detecting anomalies of vertical federated migration driven by industrial big data and a method for detecting anomalies of vertical federated migration driven by industrial big data in a system of the present invention;
[0067] Figure 4 A schematic diagram of an industrial big data driven vertical federated migration anomaly detection method and system anomaly detection accuracy results of the present invention;
[0068] Figure 5 This is a schematic diagram of a method for detecting anomalies in vertical federated migration driven by industrial big data and how the accuracy rate in the system changes with the degree of sample overlap in the present invention. DETAILED DESCRIPTION
[0069] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0070] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0071] How to ensure that all industrial raw data can be retained locally by the participants to avoid data leakage, and how to improve the accuracy of detection.
[0072] Based on this, the present invention provides an industrial big data driven vertical federation migration anomaly detection method, comprising the following steps:
[0073] S1. Obtain a source domain dataset and a target domain dataset from an industrial scenario; wherein the source domain dataset is constructed based on industrial data with known abnormal labels, and the target domain dataset is constructed based on industrial data without abnormal labels;
[0074] S2. Based on the preset vertical federation migration model:
[0075] Perform vertical federated feature extraction: map the source domain dataset and the target domain dataset into a common feature space to obtain latent features; perform domain adaptation: extract domain-invariant and distinguishable features from latent features; perform joint domain alignment: align the distance between domains, map the domain-invariant and distinguishable features to obtain the final anomaly label;
[0076] Among them, the preset vertical federation migration model is constructed through the vertical federation feature extraction module, the domain adaptation module and the joint domain alignment module, and then obtained through training;
[0077] Among them, the training is carried out based on the unsupervised longitudinal federated migration model pre-training process and the longitudinal federated migration model training process.
[0078] The method of the present invention can migrate the anomaly detection model from a highly heterogeneous source domain to a target domain with zero anomaly label samples, thereby avoiding data leakage and improving the accuracy of detection. Specifically:
[0079] 1) Dataset construction;
[0080] The data with known abnormal labels are selected to construct the source domain dataset, and the target domain participants use their local data without abnormal labels to construct the target domain dataset respectively; wherein the data distribution of the source domain dataset and the target domain dataset are different from each other;
[0081] 2) Construction of vertical federation migration model;
[0082] The vertical federated migration model mainly includes three modules: vertical federated feature extraction, domain adaptation and joint domain alignment. The vertical federated feature extraction module maps the source domain and target domain data with heterogeneous feature spaces to the common feature space to obtain latent features; the domain adaptation module reduces the difference in feature distribution in different domains and further extracts features with domain invariance and distinguishability from the latent features of the source domain and the target domain; the joint domain alignment module maps the features with domain invariance and distinguishability into the final anomaly labels;
[0083] 3) Model training;
[0084] The unsupervised federated model training process includes two steps: federated model pre-training and federated model training. The federated model pre-training adopts the local training and fine-tuning method of the source domain participants. Then the federated model training trains the unsupervised federated model based on the local and central server joint training method. After multiple iterations of training, the error curve tends to be stable and the model training is completed.
[0085] 4) Anomaly detection;
[0086] Input the unlabeled data to be tested in the target domain into the trained model to obtain the test results.
[0087] In the industrial big data driven vertical federation migration anomaly detection method of the present invention:
[0088] The vertical federated feature extraction module uses a convolutional neural network structure with a spatial pyramid pooling layer to process heterogeneous features in various fields, ensuring that heterogeneous data will output feature vectors of the same dimension after passing through the convolution layer and the spatial pyramid pooling layer, thus transforming the heterogeneous transfer learning problem into a homogeneous transfer learning problem.
[0089] The domain adaptation module is applicable to the sample heterogeneity problem and is used to extract domain invariant features from heterogeneous features and map the heterogeneous source and target domain features into a common latent feature space. The vertical federation joint domain adversarial adaptation module calculates the abnormal label classification loss and the latent feature alignment loss based on the adversarial CDAN. CDAN is a conditional domain discriminator whose loss function is the cross covariance of feature representation and classifier prediction.
[0090] The joint domain alignment module is used to improve the performance of the feature extractor in extracting domain-invariant features. The difference-based alignment method is added as a joint domain alignment to calculate the output label alignment loss, ultimately achieving the purpose of domain feature alignment.
[0091] The federated model pre-training process, used to reduce model training time and speed up model training convergence, is a preliminary improvement made by pre-training and fine-tuning methods to prevent model overfitting.
[0092] The federated model training process is suitable for implementing anomaly detection tasks while protecting the data privacy of each participant, and updating model parameters based on the joint training method of local and central servers.
[0093] Furthermore, in the present invention, the specific operation of the vertical federation feature extraction module is:
[0094] Add a spatial pyramid pooling layer before the fully connected layer of the convolutional neural network. The spatial pyramid pooling layer consists of multiple adaptive convolutional layers at different levels. Unlike the traditional convolutional layer that fixes the convolution kernel, padding, and stride parameters to obtain output features, adaptive convolution automatically adjusts the shape of the convolution kernel, padding, and stride parameters to adapt to inputs of different sizes by fixing the size of the output feature map. Assuming that the size of the input data and the shape of the output feature of the adaptive convolution layer are known, the shape and size of the convolution kernel, padding, and stride can be inferred by the following formula:
[0095]
[0096] kernel=H in -(H out -1)*stride(2);
[0097] padding=0 (3);
[0098] Among them, stride represents the stride, padding represents the padding, and kernel represents the convolution kernel. Indicates rounding down, H in Indicates the size of the input data, H out Indicates the feature output size.
[0099] To ensure that all heterogeneous features can capture sufficient receptive field information through the spatial pyramid layer, the spatial pyramid pooling layer contains n adaptive convolutional layers of different sizes. These adaptive convolutional layers process data in parallel and finally summarize all feature information.
[0100] Furthermore, in the present invention, the domain adaptation module specifically operates as follows:
[0101] In order to map the heterogeneous source and target domain feature spaces into a latent common space, it is necessary to calculate the abnormal label classification loss and the latent feature alignment loss of the source domain dataset.
[0102] The data of the source domain is input into the feature extractor of the source domain and the output is the potential feature f S , and then obtain f S After inputting the classifier, we get the classification result g S Similarly, the data of the target domain is input into the feature extractor on the target domain to obtain the potential feature f T , and then obtain f T Input the classifier and get the classification result g T .
[0103] Calculating Domain Adaptation Loss Function
[0104]
[0105] Among them, E represents the expectation, f S and g S They represent the potential features of the source domain and the classifier results output by the potential features, respectively. T and g T Corresponding to the latent features in the target domain and the classifier results output by the latent features; Represents a multilinear map, that is, the outer product of multiple random vectors. Using cross covariance To represent the joint distribution P(x,y) of any two random vectors x and y, where Φ represents the reproducible kernel function.
[0106] At the same time, in order to avoid negative samples affecting training, an additional dynamic sample weight method is used. The update method of sample weight is as follows:
[0107]
[0108] Among them, p c Represents the classifier's final predicted probability for category c;
[0109] After the objective function is calculated, it is optimized. By minimizing the objective function, the parameters of the classifier C and the feature extractor F are optimized, while maximizing the loss to optimize the domain discriminator D. The objective function of each model is as follows:
[0110]
[0111] Where t represents the tth iteration update; and They represent the parameters of the feature extractor F, classifier C, and domain discriminator D of the t-th training respectively;
[0112] Finally, in order to avoid the classification bias of the classifier, it is also necessary to calculate the objective function of the supervised classification task on the source domain:
[0113]
[0114] in, and Represents the i-th data sample and the corresponding label in the source domain respectively; N S The source domain D S The total number of samples; F S Represents the feature extractor on the source domain;
[0115] Furthermore, the specific operation of the joint domain alignment module in the present invention is:
[0116] In order to minimize the distance between the source label distribution and the target classification result distribution, a difference-based joint alignment method is used as the joint domain alignment module to calculate the output label alignment loss.
[0117] The data of the source domain is input into the feature extractor of the source domain and the output is the potential feature f S , and then obtain f S After inputting the classifier, we get the classification result g S Similarly, the data of the target domain is input into the feature extractor on the target domain to obtain the potential feature f T , and then obtain f T Input the classifier and get the classification result g T .
[0118] Calculate the joint domain label alignment loss function:
[0119]
[0120] in, The latent features representing the source domain input of the classifier are represented at the output of layer l, It means that each layer output of the classifier is projected into a Hilbert space through a multidimensional linear mapping, |L| represents the number of layers in the classifier, and the last two layers of the classifier output are selected for alignment of anomaly detection tasks on different domains.
[0121] Furthermore, in the present invention, the specific steps of the federated model pre-training process are:
[0122] In order to reduce training time and speed up training convergence, model initialization pre-training is performed first.
[0123] Federated model pre-training first inputs source domain data, randomly initializes source domain model parameters, then calculates the following loss, and back-propagates to update source domain feature extractor and classifier parameters.
[0124]
[0125] Where μ is the learning rate; Represents the gradient information of the loss objective function.
[0126] Furthermore, in the present invention, the specific steps of the federated model training process are:
[0127] Source domain processing: Input source domain data into feature extractors and classifiers, obtain potential features and abnormal label classification losses, send features and gradient information to the central server, wait for the central server to return update information, and update feature extractor and classifier parameters. Feature extractor and classifier parameter update method:
[0128]
[0129]
[0130] Among them, λ and β are hyperparameters; f S =F S (X S ) represents the latent features extracted from the source domain.
[0131] Target domain processing: Input the target domain data into the feature extractor and classifier, obtain the potential features and abnormal label classification loss, send the features and gradient information to the central server, wait for the central server to return the update information, and update the feature extractor and classifier parameters according to formulas (15) and (16).
[0132]
[0133] The central server processes: accepts the potential features uploaded by all participants, as well as the labels corresponding to the source domain, forward propagates to calculate the loss, back propagates to obtain the gradient, and transmits the gradient information to the source domain and the target domain. The objective function that the central server needs to calculate is as follows:
[0134]
[0135] in, The classification loss of abnormal labels representing the source domain; represents latent feature alignment loss; stands for joint domain label alignment loss.
[0136] The central server back-propagates the updated parameters of the feature extractor and classifier and then passes them to the source and target domains. The network parameters are updated during training using the Adam optimizer, and the adversarial network optimization problem is solved using the gradient reversal layer.
[0137] The following is a further detailed description of an industrial big data driven vertical federated migration anomaly detection method and system of the present invention through specific embodiments.
[0138] like Figure 1 As described above, a general industrial anomaly detection scenario based on vertical federated transfer learning is presented.
[0139] Industrial anomaly detection methods based on vertical federated transfer learning can help participants with limited samples build anomaly detection models while protecting data privacy. Due to problems such as extreme data heterogeneity and missing anomaly labels, existing federated transfer learning methods cannot be specifically applied in real scenarios.
[0140] Extreme heterogeneity: Traditional federated transfer learning methods based on machine learning require overlapping samples (i.e., both the source domain and the target domain have the same set of samples) to achieve effective model migration. Since it is impossible for two parties to have overlapping samples without exchanging original data, the above assumption is difficult to guarantee in actual industrial scenarios. Although federated transfer learning methods based on deep learning do not require overlapping samples, they assume that the source domain and the target domain have the same feature space, that is, a horizontal federated transfer learning scenario, in which the participants use exactly the same sensors to collect data. However, in actual industrial scenarios, due to different monitoring settings, the feature spaces of data from different participants are often multi-scale and have strong feature space and data distribution heterogeneity.
[0141] Abnormal label missing: Most transfer learning methods require a small number of labeled samples in the target domain to fine-tune the transferred model. However, for the participants in the actual scenario, such as those factories that newly deploy equipment, it is very likely that abnormal labels do not exist. Based on the above situation, the present invention proposes an industrial big data driven vertical federated migration anomaly detection method.
[0142] like Figure 2 As shown, the present invention uses a vertical federation feature extraction module to solve the problem of heterogeneous features in vertical federation scenarios. A domain adaptation module is used to map heterogeneous source domain and target domain features to a common latent feature space. In order to enhance the transfer effect of the zero anomaly label model, a novel joint domain alignment method is constructed using a joint domain alignment module to minimize the distance between the source label distribution and the target classification result distribution.
[0143] like Figure 3 As shown, the entire unsupervised federated training process is divided into two parts: federated model pre-training and federated model training;
[0144] In order to reduce training time and speed up training convergence, model initialization pre-training is performed first.
[0145] Federated pre-training first inputs source domain data, randomly initializes source domain model parameters, then calculates losses, and updates source domain feature extractor and classifier parameters. Then comes the federated model training process:
[0146] Source domain and target domain processing: Input the source domain and target domain data into the feature extractor and classifier, obtain the aligned latent features and abnormal label classification loss, send the features and gradient information to the central server, wait for the central server to return the update information, and update the feature extractor and classifier parameters.
[0147] Central server processing: accepts potential features uploaded by all participants and labels corresponding to the source domain, forward propagates to calculate the loss, backpropagates to obtain the gradient, and transmits the gradient information to the source domain and the target domain.
[0148] like Figure 4 and Figure 5 As shown, through comparative testing, under various vertical federated heterogeneous migration task settings, the accuracy of the industrial big data driven vertical federated migration anomaly detection method of the present invention is ahead of other methods.
[0149] like Figure 4 As shown, the average accuracy of multiple tasks of the method of the present invention reached 93.56%, which is 37.64% higher than the baseline method, 46.02% higher than the Coral method, 40.82% higher than the Mk-MMD method, 53.42% higher than the CDA_E method, 78.46% higher than the DANN method, and 29% higher than the SFL method.
[0150] like Figure 5 As shown, in the experiment of accuracy changing with sample overlap, the method of the present invention is always higher than other methods, and the accuracy increases with the increase of sample overlap, reaching 88.1% when the sample overlap is 0% and 93.5% when the overlap is 100%.
[0151] In summary, the present invention provides an industrial big data-driven vertical federated migration anomaly detection method and system, which sends the joint distribution of feature and label information to the conditional domain discriminator for domain discrimination. On the one hand, the joint distribution information is used to more easily confuse the discriminator and reduce the difference in data distribution between domains. On the other hand, the feature extractor is strengthened to extract features according to semantics to help the classifier perform anomaly detection tasks. The present invention combines the label information corresponding to the data, uses the joint distribution information of data features and label information for domain adversarial training, and adds a federal pre-training algorithm at the beginning of training, which effectively alleviates the problem of heterogeneous distribution of data between domains, reduces the negative migration problem and improves the anomaly detection effect. At the same time, in order to overcome the difficulty of training the domain adversarial adaptive method, the present invention proposes that joint domain adaptation will effectively overcome the drawbacks of domain adversarial networks and alleviate the difference in joint data distribution between domains. Finally, the present invention combines these two methods to alleviate the difference in data distribution between domains while reducing the difference in the joint data distribution of sample potential features and abnormal labels, and improves the similarity of data distribution between domains to solve the anomaly detection problem.
[0152] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any ordinary technician in the industry can smoothly implement the present invention as shown in the specification and described above. However, any equivalent changes, modifications and evolutions made by technicians familiar with the profession without departing from the scope of the technical solution of the present invention using the technical content disclosed above are all equivalent embodiments of the present invention. At the same time, any equivalent changes, modifications and evolutions made to the above embodiments based on the essential technology of the present invention are still within the protection scope of the technical solution of the present invention.
Claims
1. A method for detecting anomalies in vertical federated migration driven by industrial big data, characterized in that: The steps include: S1. Obtain a source domain dataset and a target domain dataset from an industrial scenario; wherein the source domain dataset is constructed based on industrial data with known abnormal labels, and the target domain dataset is constructed based on industrial data without abnormal labels; S2. Based on the preset vertical federation migration model: Perform vertical federated feature extraction: map the source domain dataset and the target domain dataset into a common feature space to obtain latent features; perform domain adaptation: extract domain-invariant and distinguishable features from latent features; perform joint domain alignment: align the distance between domains, map the domain-invariant and distinguishable features to obtain the final anomaly label; Among them, the preset vertical federation migration model is constructed through the vertical federation feature extraction module, the domain adaptation module and the joint domain alignment module, and then obtained through training; Among them, the training is carried out based on the unsupervised longitudinal federated migration model pre-training process and the longitudinal federated migration model training process.
2. The method for detecting anomalies in vertical federated migration driven by industrial big data according to claim 1 is characterized in that: In S2, vertical federated feature extraction is based on convolutional neural networks and spatial pyramid methods to map the source domain dataset and the target domain dataset into a common feature space.
3. The method for detecting anomalies in vertical federated migration driven by industrial big data according to claim 2 is characterized in that: In S2, the specific operations of vertical federated feature extraction are: By adding a spatial pyramid pooling layer before the fully connected layer of the convolutional neural network, the spatial pyramid pooling layer includes adaptive convolution layers of different levels. The adaptive convolution automatically adjusts the shape of the convolution kernel, padding, and stride parameters by fixing the size of the output feature map. The shape and size of the convolution kernel, padding, and stride are inferred by the following formulas (1), (2), and (3). The adaptive convolution layer forms potential features. kernel=H in -(H out -1)*stride (2); padding=0 (3); Among them, stride represents the stride, padding represents the padding, and kernel represents the convolution kernel. Indicates rounding down, H in Indicates the size of the input data, H out Indicates the feature output size.
4. The method for detecting anomalies in vertical federated migration driven by industrial big data according to claim 1, characterized in that: In S2, domain adaptation is based on the principle of conditional domain adversarial learning to reduce the differences in feature distributions in different domains and extract domain-invariant and distinguishable features from latent features.
5. The method for detecting anomalies in vertical federated migration driven by industrial big data according to claim 4 is characterized in that: In S2, the specific operations of domain adaptation are: The latent feature alignment loss and the abnormal label classification loss of the source domain dataset are optimized based on the adversarial learning principle; on the one hand, the objective function of the latent feature alignment loss is as follows: Among them, E represents the expectation, f s and g S They represent the potential features of the source domain and the classifier results output by the potential features, respectively. T and g T Corresponding to the latent features in the target domain and the classifier results output by the latent features; represents a multilinear map; The sample weight method of formula (5) below is used to avoid negative samples from affecting training: Among them, p represents the final prediction probability of the classifier for each category; The optimization method based on joint domain adversarial learning is used to optimize the latent feature alignment loss; by minimizing the objective function of the latent feature alignment loss, the parameters of the feature extractor F and the classifier C are optimized, while maximizing the objective function of the latent feature alignment loss to optimize the parameters of the domain discriminator D. The optimization process of each model parameter is as follows: Among them, t represents the tth iteration update, and They represent the parameters of the feature extractor F, classifier C, and domain discriminator D of the t-th training respectively; On the other hand, the objective function of the abnormal label classification loss of the source domain dataset is as follows: in, and Represents the i-th data sample and the corresponding label in the source domain respectively; N S The source domain D S The total number of samples; F S Represents the feature extractor on the source domain.
6. The method for detecting anomalies in vertical federated migration driven by industrial big data according to claim 1, characterized in that: In S2, joint domain alignment is a difference-based label distribution distance alignment method, which maps domain-invariant and distinguishable features into final anomaly labels, achieving anomaly detection in the target domain without anomaly labels.
7. The method for detecting anomalies in vertical federated migration driven by industrial big data according to claim 6, characterized in that: In S2, the specific operation of joint domain alignment is: By adding the difference-based label distribution distance alignment method as a joint domain alignment to calculate the output label alignment loss, the objective function of the joint domain label alignment loss is as follows: in, The latent features representing the source domain input in the classification network are represented at the output of layer l, It means that each layer output of the classifier is projected into a Hilbert space through a multidimensional linear mapping, |L| represents the number of layers in the classifier, and the last two layers of the classifier output are selected for alignment of anomaly detection tasks on different domains.
8. The method for detecting anomalies in vertical federated migration driven by industrial big data according to claim 1, characterized in that: In S2, the model pre-training process is transferred vertically and federated unsupervisedly to reduce the training time and speed up the training convergence. Specifically: The model input is the source domain dataset, the source domain feature extractor and classifier parameters are randomly initialized, and then the objective function of the pre-training loss of the longitudinal federated migration model is calculated as follows: Then back propagation updates the source domain feature extractor and classifier parameters as follows: Where μ is the learning rate; Represents the gradient information of the loss objective function.
9. The method for detecting anomalies in vertical federated migration driven by industrial big data according to claim 1, characterized in that: In S2, the vertical federation migration model training process trains the vertical federation migration model based on the local and central server joint training method, specifically: The model input is the source domain dataset and the target domain dataset. The model parameters are initialized to the model parameters obtained after pre-training the longitudinal federated migration model. The source domain dataset is input into the feature extractor and classifier to obtain the potential features and abnormal label classification loss. The potential features and gradient information are sent to the central server. The central server is waited for to return the update information. The feature extractor and classifier parameters are updated by the following formula: Among them, λ and β are hyperparameters; f S =F S (X S ) represents the latent features extracted from the source domain; Randomly select target domain samples in the target domain and obtain the latent features f after the feature extractor T And the objective function, send the potential features to the central server, block waiting for parameter updates from the central server, and update the feature extractor and classifier parameters through the following formula: Among them, μ represents the learning rate, and t represents the tth iteration update; The central server receives all uploaded potential features and the corresponding labels of the source domain, performs reverse labeling, and calculates the objective function as follows: in, Represents the abnormal label classification loss of the source domain dataset; represents latent feature alignment loss; stands for joint domain label alignment loss; The central server forward propagates to calculate the loss, backpropagates to obtain the gradient, and transmits the gradient information to the source domain and the target domain.
10. An industrial big data driven vertical federated migration anomaly detection system, characterized in that: It includes a vertical federated feature extraction module, a domain adaptation module, and a joint domain alignment module; among which: A vertical federated feature extraction module is used to map the source domain dataset and the target domain dataset into a common feature space to obtain latent features; The domain adaptation module is used to extract domain-invariant and distinguishable features from latent features; The joint domain alignment module is used to align the distances between domains and map the features with domain invariance and distinguishability to obtain the final anomaly labels.
Citation Information
Cited By
Radioactive geophysical exploration data processing method and system based on physical field attenuation model
CN119738889A
Product package quality inspection classification method based on federal transfer learning
CN119884838A
Silica gel button conductivity reinforcement learning test system and silica gel button conductivity reinforcement learning test method
CN120671763A
Silicone button conductive property reinforcement learning test system and method
CN120671763B
Mechanical fault diagnosis method and system based on federal field generalization
CN121094060A
