Industrial big data-driven vertical federated transfer-based anomaly detection method and system

Abstract

An industrial big data-driven vertical federated transfer-based anomaly detection method and system. The method comprises: acquiring a source domain data set and a target domain data set from an industrial scenario, the source domain data set being constructed on the basis of industrial data having known anomaly labels, and the target domain data set being constructed on the basis of industrial data without anomaly labels; and on the basis of a preset vertical federated transfer model: performing vertical federated feature extraction: mapping the source domain data set and the target domain data set into a common feature space to obtain potential features; performing domain adaptation: extracting features having domain invariance and discriminability from among the potential features; and performing joint domain alignment: aligning the distance between domains, and mapping the features having domain invariance and discriminability to obtain the anomaly labels.

Description

Industrial big data driven longitudinal federal migration anomaly detection method and system TECHNICAL FIELD

[0001] The present application belongs to the field of industrial intelligence technology, and specifically relates to an industrial big data driven longitudinal federal migration anomaly detection method and system. BACKGROUND

[0002] Industrial big data is a key production factor, which involves data of the whole life cycle of products and services in the industrial field, including data generated and used in the links of research and development design, production and manufacturing, operation and management, operation and service, and data in the industrial internet platform. The collection method of industrial big data is mainly based on sensors, supplemented by manual uploading, covering all links of industrial production. Industrial big data also has the characteristics of complex dynamic system, including 4V characteristics of large scale, high speed, type diversity and low quality, as well as new characteristics of multi-modal, strong correlation and high throughput. These characteristics make industrial big data face a series of technical challenges in analysis and application. In order to obtain more knowledge from these massive data and improve competitiveness, machine learning has become an important technology in the industrial field.

[0003] In the analysis of industrial big data, anomaly detection and anomaly warning of data are very important. Through analysis and mining of industrial data, the abnormal state of production equipment, the abnormal fluctuation of sales process and the abnormal behavior of operation process can be detected in time, and the sudden abnormality of key links can be prevented, so as to take corresponding measures to ensure the normal operation of the whole life cycle of industry, so as to meet the development requirements of high efficiency, high yield and automation of industrial industry. The machine learning method based on data driving has been widely applied in the field of industrial anomaly detection, and the goal is to extract various features from various mode diverse and working condition variable industrial big data (such as current, voltage, vibration, temperature and acoustic emission) sensor signals, to complete the effective modeling and differentiation of abnormal mode and normal mode. The main method depends on a large number of well-labeled samples to train various machine learning (ML) models (for example, SVM, DNN), which are obviously superior to traditional anomaly detection methods based on part of system mechanism or expert experience.

[0004] However, in actual industrial scenarios, due to the extremely scarce abnormal samples owned by a large number of industrial participants, the traditional data-driven method based on a large number of labeled samples cannot be used. For example, for a factory newly deploying a device, device anomalies are usually low-probability events, so the data anomaly rate is generally low, and it usually takes a considerable amount of time to appear anomalies and collect samples with a definite "abnormal" label. How to build an effective anomaly detection model with few or even zero abnormal labels has become a bottleneck problem faced by the current industry. In order to solve this problem, under the assumption of independent and identically distributed (IID), an anomaly detection model can be trained on more labeled related samples of similar other industrial participants (i.e., source domain), and then deployed on the participant with scarce samples (i.e., target domain). However, the IID assumption is almost impossible to guarantee in real scenarios. Due to multiple differences such as monitoring settings, workloads, transmission paths, noise interference, and abnormality degree, the samples of different participants under different working conditions will inevitably have obvious distribution differences, that is, different participants usually have different sample domains. Transfer learning can be used to build and transfer models of different domains and has been widely applied to industrial anomaly detection.

[0005] However, existing transfer methods need to exchange samples between the source domain and the target domain to extract common domain invariant information. However, considering the influence of stringent regulations such as the General Data Protection Regulation (GDPR) and other related factors, the current anomaly detection method based on transfer learning cannot effectively guarantee the privacy of industrial data participants, and the accuracy of abnormal label detection still needs to be further improved.

[0006] SUMMARY

[0007] The purpose of the present application is to overcome the shortcomings of the prior art and provide a digital predistortion nonlinear model forward synthesis method and system derived from circuit description.

[0008] The present application adopts the following technical solutions to achieve the above purposes:

[0009] The purpose of the present application is to overcome the shortcomings of the prior art and provide a digital predistortion nonlinear model forward synthesis method and system derived from circuit description.

[0010] To achieve the above purposes, the present application adopts the following technical solutions:

[0011] The present application provides an industrial big data driven longitudinal federal transfer anomaly detection method, comprising the following steps:

[0012] S1, obtaining a source domain dataset and a target domain dataset from an industrial scene; wherein the source domain dataset is constructed based on industrial data with known abnormal labels, and the target domain dataset is constructed based on industrial data without abnormal labels;

[0013] S2, based on a preset longitudinal federated transfer model:

[0014] performing longitudinal federated feature extraction: mapping the source domain dataset and the target domain dataset into a common feature space to obtain latent features; performing domain adaptation: extracting features with domain invariance and distinguishability from the latent features; performing joint domain alignment: aligning the distance between domains, and mapping the features with domain invariance and distinguishability to obtain final abnormal labels;

[0015] wherein the preset longitudinal federated transfer model is constructed through a longitudinal federated feature extraction module, a domain adaptation module and a joint domain alignment module, and is obtained through training;

[0016] wherein the training is based on an unsupervised longitudinal federated transfer model pre-training process and a longitudinal federated transfer model training process.

[0017] In some embodiments, in S2, the longitudinal federated feature extraction is based on a convolutional neural network and a spatial pyramid method, and the source domain dataset and the target domain dataset are mapped into a common feature space.

[0018] Further, in S2, the specific operation of the longitudinal federated feature extraction is:

[0019] By adding a spatial pyramid pooling layer before the fully connected layer of the convolutional neural network, the spatial pyramid pooling layer includes adaptive convolution layers of different levels, the adaptive convolution automatically adjusts the shapes of the convolution kernel, padding and stride by fixing the size of the output feature map, and the shapes of the convolution kernel, padding and stride are back calculated through the following formulas (1), (2), (3), and the adaptive convolution layer forms latent features; kernel=H in -(H out -1)*stride (2); padding=0 (3);

[0020] wherein stride represents stride, padding represents padding, kernel represents convolution kernel, represents rounding down, H in represents the size of the input data, H out represents the size of the feature output.

[0021] In some embodiments, in S2, the domain adaptation is based on the conditional domain adversarial learning principle to reduce the difference between different domain feature distributions, and features with domain invariance and distinguishability are extracted from the latent features.

[0022] Further, in S2, the specific operation of the domain adaptation is:

[0023] By optimizing the latent feature alignment loss and the abnormal label classification loss of the source domain dataset based on the adversarial learning principle, on the one hand, the objective function of the latent feature alignment loss is as follows:

[0024] Wherein, E represents the expectation, f S and g S respectively represent the latent features of the source domain and the classifier results output by the latent features, f T and g T correspond to the latent features in the target domain and the classifier results output by the latent features; represents a multilinear mapping;

[0025] The sample weight method of formula (5) is used to avoid the influence of negative samples on training:

[0026] Wherein, p represents the prediction probability of the classifier for each class;

[0027] The latent feature alignment loss is optimized based on the optimization method of the joint domain adversarial learning; by minimizing the objective function of the latent feature alignment loss, the parameters of the feature extractor F and the classifier C are optimized, and the parameters of the domain discriminator D are optimized by maximizing the objective function of the latent feature alignment loss, and the optimization process of each model parameter is as follows:

[0028] Wherein, t represents the tthiteration update, and respectively represent the parameters of the feature extractor F, the classifier C and the domain discriminator D in the tthtraining;

[0029] On the other hand, the objective function of the abnormal label classification loss of the source domain dataset is as follows:

[0030] Wherein, and respectively represent the i thdata sample and the corresponding label on the source domain; N S is the total number of samples of the source domain D S ; F S represents the feature extractor on the source domain.

[0031] In some embodiments, in S2, the joint domain alignment is a difference-based label distribution distance alignment method, which maps features with domain invariance and distinguishability to final anomaly labels, achieving anomaly detection in anomaly-free target domains.

[0032] Further, in S2, the specific operation of joint domain alignment is:

[0033] By adding a difference-based label distribution distance alignment method as the joint domain alignment, the output label alignment loss is calculated, and the objective function of the joint domain label alignment loss is as follows:

[0034] wherein, represents the output of the latent feature representation of the source domain input in the classification network at the lth layer, represents that the output of each layer of the classifier is projected to a Hilbert space through a multi-dimensional linear mapping, |L| represents the number of layers in the classifier, and the last two layers of the classifier output are selected for alignment on different domains for the anomaly detection task.

[0035] In some embodiments, in S2, an unsupervised longitudinal federated transfer model pre-training process is used to reduce training time and speed up training convergence; specifically:

[0036] The model input is the source domain dataset, and the source domain feature extractor and classifier parameters are randomly initialized, and then the objective function of the longitudinal federated transfer model pre-training loss is calculated, as shown in the following formula (10):

[0037] Then the source domain feature extractor and classifier parameters are updated by backpropagation, as shown in the following formulas (11) and (12):

[0038] wherein μ is the learning rate; represents the gradient information of the loss objective function.

[0039] In some embodiments, in S2, the longitudinal federated transfer model training process trains the longitudinal federated transfer model based on a local and central server joint training method, specifically:

[0040] The model input is the source domain dataset and the target domain dataset, the model parameters are initialized to the model parameters obtained after the longitudinal federated transfer model pre-training, the source domain dataset is input into the feature extractor and the classifier, the latent features and anomaly label classification loss are obtained, the latent features and gradient information are sent to the central server, and the central server returns the update information, and the feature extractor and classifier parameters are updated by the following formula:

[0041] wherein λ and β are hyperparameters; f S =F S (X S ) represents the extracted latent feature on the source domain;

[0042] a target domain sample is randomly selected in the target domain, the latent feature f T and the target function are obtained through the feature extractor, the latent feature is sent to the central server, the parameter update from the central server is blocked and waits, and the feature extractor and the classifier parameter are updated through the following formula:

[0043] wherein μ represents a learning rate, and t represents the tth iteration update;

[0044] The central server receives all uploaded latent features and corresponding labels of the source domain, carries out reverse labeling, and calculates the target function as follows:

[0045] wherein, represents the abnormal label classification loss of the source domain data set; represents the latent feature alignment loss; represents the joint field label alignment loss;

[0046] The central server calculates the loss by forward propagation, obtains the gradient by backward propagation, and transmits the gradient information to the source domain and the target domain.

[0047] The application also provides an industrial big data driven longitudinal federal transfer anomaly detection system, comprising a longitudinal federal feature extraction module, a field adaptation module and a joint field alignment module; wherein:

[0048] The longitudinal federal feature extraction module is used for mapping the source domain data set and the target domain data set into a public feature space to obtain a latent feature;

[0049] The field adaptation module is used for extracting features with domain invariance and distinguishability from the latent feature;

[0050] The joint field alignment module is used for aligning the distance between the fields and mapping the features with domain invariance and distinguishability to obtain the final abnormal label.

[0051] Compared with the prior art, the industrial big data driven longitudinal federal transfer anomaly detection method and system have the following beneficial technical effects.

[0052] The industrial big data driven longitudinal federal migration anomaly detection method comprises the following steps: S1, obtaining a source domain data set and a target domain data set from an industrial scene; wherein the source domain data set is constructed based on known abnormal labels of industrial data, and the target domain data set is constructed based on industrial data without abnormal labels; S2, based on a preset longitudinal federal migration model: longitudinal federal feature extraction: mapping the source domain data set and the target domain data set to a common feature space to obtain latent features; domain adaptation: extracting features with domain invariance and distinguishability from the latent features; joint domain alignment: aligning the distance between domains and mapping the features with domain invariance and distinguishability to obtain final abnormal labels; wherein the preset longitudinal federal migration model is constructed through a longitudinal federal feature extraction module, a domain adaptation module and a joint domain alignment module, and is obtained through training; wherein the training is based on an unsupervised longitudinal federal migration model pre-training process and a longitudinal federal migration model training process. Based on the above, in the actual industrial scene with heterogeneous data and missing abnormal labels, the unsupervised longitudinal federal migration method is used to help the participants with missing abnormal labels to construct an anomaly detection model under the premise of protecting data privacy and security. The present application can realize cross-participant federal migration learning based on the federal learning architecture, so that all original data can be kept in the local of the participants, thereby avoiding data leakage. The present application maps heterogeneous features on different domains to a common feature space through the construction of a longitudinal federal feature extraction module; the domain adaptation module is constructed to reduce the difference in feature distribution on different domains, and further extract features with domain invariance and distinguishability; the joint domain alignment module is constructed to align the distance between domains, realize anomaly detection in the target domain without abnormal labels; the unsupervised federal model training process does not require the participants to share data, accelerates the model training convergence speed and prevents model overfitting. The present application has high accuracy and strong generalization, and can meet the application requirements of anomaly detection in actual industrial processes. BRIEF DESCRIPTION OF DRAWINGS

[0053] The accompanying drawings are included to provide a further understanding of the present application, and are incorporated in and constitute a part of the specification. The illustrative embodiments of the present application and their description serve to explain the present application. They do not, however, limit the present application, which is defined solely by the appended claims.

[0054] Fig. 1 is a model schematic diagram of the industrial big data driven longitudinal federal migration anomaly detection method and system applied to an industrial anomaly detection scene;

[0055] Fig. 2 is a basic architecture schematic diagram of the unsupervised longitudinal federal migration anomaly detection model in the industrial big data driven longitudinal federal migration anomaly detection method and system;

[0056] Fig. 3 is a schematic diagram of the training basic process of the industrial big data driven longitudinal federal migration anomaly detection method in the industrial big data driven longitudinal federal migration anomaly detection method and system according to the present application;

[0057] Fig. 4 is a schematic diagram of the anomaly detection accuracy result of the industrial big data driven longitudinal federal migration anomaly detection method and system according to the present application;

[0058] Fig. 5 is a schematic diagram of the accuracy change with the sample overlap degree in the industrial big data driven longitudinal federal migration anomaly detection method and system according to the present application. DETAILED DESCRIPTION

[0059] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should fall within the scope of protection of the present application.

[0060] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0061] How to enable all industrial raw data to be retained in the local of the participant, thereby avoiding data leakage, and how to improve the accuracy of detection.

[0062] Based on this, the present application provides an industrial big data driven longitudinal federal migration anomaly detection method, comprising the following steps:

[0063] S1, obtaining a source domain data set and a target domain data set from an industrial scene; wherein the source domain data set is constructed based on industrial data with known abnormal labels, and the target domain data set is constructed based on industrial data without abnormal labels;

[0064] S2, based on a preset longitudinal federal migration model:

[0065] Longitudinal federal feature extraction is performed: the source domain dataset and the target domain dataset are mapped into a common feature space to obtain latent features; domain adaptation is performed: features with domain invariance and distinguishability are extracted from the latent features; joint domain alignment is performed: the distance between the domains is aligned, and the features with domain invariance and distinguishability are mapped to obtain final anomaly labels;

[0066] The preset longitudinal federal transfer model is constructed through the longitudinal federal feature extraction module, the domain adaptation module and the joint domain alignment module, and is obtained through training.

[0067] The training is based on the unsupervised longitudinal federal transfer model pre-training process and the longitudinal federal transfer model training process.

[0068] The method of the application can migrate the anomaly detection model from a highly heterogeneous source domain to a target domain with zero anomaly label samples, thereby avoiding data leakage and improving the accuracy of detection. Specifically:

[0069] 1) Dataset construction;

[0070] The source domain dataset is constructed by selecting data with known anomaly labels, and the target domain participants construct target domain datasets respectively using their local data without anomaly labels; wherein the data distribution of the source domain dataset and the target domain dataset is different from each other;

[0071] 2) Longitudinal federal transfer model construction;

[0072] The longitudinal federal transfer model mainly includes three modules of longitudinal federal feature extraction, domain adaptation and joint domain alignment, wherein the longitudinal federal feature extraction module maps the source domain and the target domain data with heterogeneous feature spaces into a common feature space to obtain latent features; the domain adaptation module reduces the difference between the feature distributions of different domains, and further extracts features with domain invariance and distinguishability from the latent features of the source domain and the target domain; the joint domain alignment module maps the features with domain invariance and distinguishability to final anomaly labels;

[0073] 3) Model training;

[0074] The unsupervised federal model training process includes two steps of federal model pre-training and federal model training, wherein the source domain participant locally trains the pre-training fine-tuning method, and then the federal model training trains the unsupervised federal model based on the joint training method of the local and central server; after multiple iterations of training, the error curve tends to be stable, and the model training is completed.

[0075] 4) Anomaly detection;

[0076] The unlabeled data to be detected of the target domain are input into the trained model to obtain the detection result.

[0077] In an industrial big data driven longitudinal federal migration anomaly detection method of the application, the longitudinal federal feature extraction module uses a convolutional neural network structure containing a spatial pyramid pooling layer to process heterogeneous features in each field, so that after the heterogeneous data passes through the convolutional layer and the spatial pyramid pooling layer, the same dimension feature vector is output, and the heterogeneous transfer learning problem is converted into a homogeneous transfer learning problem.

[0078] The longitudinal federal feature extraction module uses a convolutional neural network structure containing a spatial pyramid pooling layer to process heterogeneous features in each field, so that after the heterogeneous data passes through the convolutional layer and the spatial pyramid pooling layer, the same dimension feature vector is output, and the heterogeneous transfer learning problem is converted into a homogeneous transfer learning problem.

[0079] The domain adaptation module is suitable for sample heterogeneity problems, and is used for extracting domain invariant features from heterogeneous features, and mapping heterogeneous source domain and target domain features to a common latent feature space. The longitudinal federal joint domain alignment adaptation module calculates the anomaly label classification loss and the latent feature alignment loss based on the CDAN of the confrontation. CDAN is a conditional domain discriminator, and the loss function is the cross-covariance of the feature representation and the classifier prediction.

[0080] The joint domain alignment module is used to improve the performance of the domain invariant features extracted by the feature extractor, and adds a difference-based alignment method as the joint domain alignment to calculate the output label alignment loss, so as to achieve the purpose of domain feature alignment.

[0081] The federal model pre-training process is used to reduce the model training time and speed up the model training convergence speed, and is a preliminary improvement by using the pre-training and fine-tuning methods to prevent model overfitting.

[0082] The federal model training process is suitable for realizing the anomaly detection task under the premise of protecting the data privacy of each participant, and updates the model parameters based on the local and central server joint training method.

[0083] Further, in the longitudinal federal feature extraction module of the application, the specific operation is as follows:

[0084] A spatial pyramid pooling layer is added before the fully connected layer of the convolutional neural network. The spatial pyramid pooling layer is composed of multiple adaptive convolutional layers of different levels. Unlike the traditional convolutional layer which fixes the convolution kernel, padding and stride parameters to obtain the output features, the adaptive convolution automatically adjusts the shape of the convolution kernel, padding and stride parameters to adapt to different sizes of the input by fixing the size of the output feature map. Assuming that the size of the input data and the shape of the adaptive convolutional layer output feature are known, the shape size of the convolution kernel (kernel), padding (padding) and stride (stride) can be back calculated by the following formula: kernel=H in -(H out -1)*stride (2); padding=0 (3);

[0085] where stride denotes the stride, padding denotes padding, and kernel denotes the convolution kernel, denotes the floor function, H in denotes the size of the input data, H out denotes the feature output size.

[0086] To ensure that all heterogeneous features can capture sufficient receptive field information through the spatial pyramid layer, the spatial pyramid pooling layer includes n adaptive convolution layers of different sizes, which process data in parallel and finally unify all feature information.

[0087] Further, in the present application, the domain adaptation module specifically operates as follows:

[0088] In order to map the heterogeneous source domain and target domain feature space to a common latent space, it is necessary to calculate the abnormal label classification loss of the source domain dataset and the latent feature alignment loss.

[0089] The data input of the source domain is output by the feature extractor of the source domain to obtain the latent feature f S , and f S is input into the classifier to obtain the classification result g S . Similarly, the data of the target domain is input into the feature extractor of the target domain to obtain the latent feature f T , and f T is input into the classifier to obtain the classification result g T .

[0090] The domain adaptation loss function is calculated

[0091] where E denotes the expectation, f S and g S denote the latent feature of the source domain and the classifier result output by the latent feature, respectively, f T and g T correspond to the latent feature in the target domain and the classifier result output by the latent feature; denotes a multilinear mapping, i.e., the outer product of multiple random vectors. The cross-covariance is used to represent the joint distribution P(x,y) of any two random vectors x and y, where Φ denotes a reproducible kernel function.

[0092] Meanwhile, in order to avoid the influence of negative samples on training, an additional dynamic sample weight method is used. The updating method of the sample weight is as follows:

[0093] where p crepresents the final prediction probability of the classifier for class c;

[0094] After the objective function is calculated, optimization is performed. By minimizing the objective function, the parameters of the classifier C and the feature extractor F are optimized, and the loss is maximized to optimize the domain discriminator D, and the objective function of each model is as follows:

[0095] Where t represents the tth iteration update; and Respectively represent the parameters of the feature extractor F, the classifier C and the domain discriminator D of the tth training;

[0096] Finally, in order to avoid the classification bias of the classifier, the objective function of the supervised classification task on the source domain also needs to be calculated:

[0097] Where, and Respectively represent the i th data sample and the corresponding label on the source domain; N S is the total number of samples of the source domain D S ; F S represents the feature extractor on the source domain;

[0098] Further, the specific operation of the joint field alignment module in the present application is:

[0099] In order to minimize the distance between the source label distribution and the target classification result distribution, a difference-based joint alignment method is used as a joint field alignment module to calculate the output label alignment loss.

[0100] The data of the source domain is input into the feature extractor of the source domain to output the latent feature f S , and then the obtained f S is input into the classifier to obtain the classification result g S . Similarly, the data of the target domain is input into the feature extractor on the target domain to output the latent feature f T , and then the obtained f T is input into the classifier to obtain the classification result g T .

[0101] The joint field label alignment loss function is calculated:

[0102] Where, represents the output of the l th layer of the latent feature representation of the source domain input of the classifier, Each layer output of the classifier is projected to a Hilbert space by a multi-dimensional linear mapping, |L| represents the number of layers in the classifier, and the last two layers of the classifier output are selected for alignment of the anomaly detection task on different domains.

[0103] Further, in the present application, the specific steps of the federal model pre-training process are:

[0104] In order to reduce the training time and speed up the training convergence, model initialization pre-training is performed first.

[0105] The federal model pre-training first inputs the source domain data, randomly initializes the source domain model parameters, then calculates the loss as follows, and updates the source domain feature extractor and classifier parameters by back propagation

[0106] Wherein, μ is the learning rate; The gradient information of the loss objective function is represented.

[0107] Further, in the present application, the specific steps of the federal model training process are:

[0108] Source domain processing: input the source domain data into the feature extractor and the classifier to obtain the latent feature and the anomaly label classification loss, send the feature and the gradient information to the central server, wait for the central server to return the update information, and update the feature extractor and the classifier parameters.

[0109] Wherein, λ and β are hyperparameters; f S = F S (X S ) represents the extracted latent feature on the source domain.

[0110] Target domain processing: input the target domain data into the feature extractor and the classifier to obtain the latent feature and the anomaly label classification loss, send the feature and the gradient information to the central server, wait for the central server to return the update information, and update the feature extractor and the classifier parameters according to formulas (15) and (16).

[0111] Central server processing: accept all the uploaded latent features and the corresponding labels of the source domain, calculate the loss by forward propagation, obtain the gradient by back propagation, and transmit the gradient information to the source domain and the target domain.

[0112] Wherein, anomaly label classification loss representing the source domain; latent feature alignment loss representing the source domain; joint domain label alignment loss representing the source domain.

[0113] The central server back-propagates to calculate the feature extractor and classifier update parameters, and then passes them to the source domain and the target domain. The network parameters are updated during the training process using the Adam optimizer, and the gradient reversal layer is used to solve the adversarial network optimization problem.

[0114] The industrial big data driven longitudinal federal migration anomaly detection method and system will be further described in detail below through specific embodiments.

[0115] As shown in FIG. 1, a general industrial anomaly detection scenario based on longitudinal federal migration learning is presented.

[0116] The industrial anomaly detection method based on longitudinal federal migration learning can help the sample-limited participants to build an anomaly detection model under the premise of protecting data privacy. Due to the problems of extremely heterogeneous data and missing anomaly labels, existing federal migration learning methods cannot be applied in real scenarios.

[0117] Extremely heterogeneous: Traditional federal migration learning methods based on machine learning require overlapping samples (i.e., both the source domain and the target domain have the same sample set) to achieve effective model migration. Since it is impossible for two participants to have overlapping samples without exchanging raw data, the above assumption is difficult to guarantee in actual industrial scenarios. Federal migration learning methods based on deep learning do not require overlapping samples, but assume that the source domain and the target domain have the same feature space, i.e., a horizontal federal migration learning scenario in which participants use the same sensors to collect data. However, in actual industrial scenarios, due to different monitoring settings, the feature spaces of data from different participants are often multi-scale and have strong feature space and data distribution heterogeneity.

[0118] Missing anomaly labels: Most migration learning methods require a small number of labeled samples in the target domain to fine-tune the transferred model. However, for participants in real scenarios, such as factories with newly deployed equipment, it is likely that there are no anomaly labels. In view of the above, the present application proposes an industrial big data driven longitudinal federal migration anomaly detection method.

[0119] As shown in FIG. 2, the application uses a longitudinal federal feature extraction module to solve the problem of heterogeneous features in the longitudinal federal scenario. A domain adaptation module is used to map the heterogeneous source domain and target domain features to a common latent feature space. In order to enhance the transfer effect of the zero abnormal label model, a novel joint domain alignment method is constructed using the joint domain alignment module, which minimizes the distance between the source label distribution and the target classification result distribution.

[0120] As shown in FIG. 3, the entire unsupervised federal training process is divided into two parts: federal model pre-training and federal model training.

[0121] In order to reduce the training time and speed up the training convergence, model initialization pre-training is performed first.

[0122] The federal pre-training first inputs the source domain data, randomly initializes the source domain model parameters, then calculates the loss, and updates the source domain feature extractor and classifier parameters. Then is the federal model training process:

[0123] Source domain and target domain processing: input the source domain and target domain data into the feature extractor and classifier, obtain the aligned latent features and abnormal label classification loss, send the features and gradient information to the central server, and wait for the central server to return the update information to update the feature extractor and classifier parameters.

[0124] Central server processing: accept all participants' uploaded latent features, and the corresponding labels of the source domain, forward propagation to calculate the loss, and backward propagation to obtain the gradient, and send the gradient information to the source domain and target domain.

[0125] As shown in FIG. 4 and FIG. 5, through comparative detection, the accuracy of the industrial big data driven longitudinal federal transfer anomaly detection method of the application is higher than that of other methods in multiple longitudinal federal heterogeneous transfer task settings.

[0126] As shown in FIG. 4, the average accuracy of multiple tasks of the method of the application reached 93.56%, which was 37.64% higher than that of the baseline method, 46.02% higher than that of the Coral method, 40.82% higher than that of the Mk-MMD method, 53.42% higher than that of the CDA_E method, 78.46% higher than that of the DANN method, and 29% higher than that of the SFL method.

[0127] As shown in FIG. 5, in the experiment of accuracy change with sample overlap degree, the method of the application is always higher than other methods, and the accuracy increases with the increase of sample overlap degree, which is 88.1% when the sample overlap degree is 0%, and reaches 93.5% when the overlap degree is 100%.

[0128] To sum up, the industrial big data driven longitudinal federal migration anomaly detection method and system, the joint distribution of features and label information is sent into the conditional domain discriminator for domain discrimination, on the one hand, the joint distribution information is used to confuse the discriminator and reduce the difference between the data distribution of different domains, on the other hand, the feature extractor is strengthened to extract features according to the semantics to help the classifier to perform the anomaly detection task. The application combines the corresponding label information of the data, uses the joint distribution information of the data features and the label information for domain adversarial training, and increases the federal pre-training algorithm at the beginning of training, effectively alleviates the problem of data distribution heterogeneity between domains, reduces the negative transfer problem and improves the anomaly detection effect. At the same time, in order to overcome the training difficulty problem of the domain adversarial adaptive method, the application proposes joint domain adaptation, which can effectively overcome the disadvantages of the domain adversarial network and alleviate the difference between the joint data distribution of the sample potential features and the abnormal labels. Finally, the application combines the two methods to alleviate the difference between the data distribution of different domains and reduce the difference between the joint data distribution of the sample potential features and the abnormal labels, improve the similarity between the data distribution of different domains, and solve the anomaly detection problem.

[0129] Finally, it should be noted that: the above description is only the preferred embodiment of the application, and does not limit the application in any form; anyone skilled in the art can easily implement the application according to the description and the above description; however, those skilled in the art can make some changes, modifications and equivalent changes of the above-mentioned technical contents without departing from the scope of the technical solutions of the application; at the same time, any equivalent changes, modifications and evolution of the above-mentioned technical contents without departing from the scope of the technical solutions of the application, all belong to the equivalent embodiments of the application; at the same time, any equivalent changes, modifications and evolution of the above-mentioned technical contents without departing from the scope of the technical solutions of the application, all belong to the equivalent embodiments of the application; at the same time, any equivalent changes, modifications and evolution of the above-mentioned technical contents without departing from the scope of the technical solutions of the application, all belong to the equivalent embodiments of the application.

Claims

1. An industry big data driven longitudinal federated migration anomaly detection method, characterized in that, Comprising the following steps: S1, obtaining a source domain dataset and a target domain dataset from an industrial scene; wherein the source domain dataset is constructed based on industrial data with known abnormal labels, and the target domain dataset is constructed based on industrial data without abnormal labels; S2, based on a preset longitudinal federal transfer model: Longitudinal federal feature extraction: mapping the source domain dataset and the target domain dataset into a common feature space to obtain latent features; domain adaptation: extracting features with domain invariance and distinguishability from the latent features; joint domain alignment: aligning the distance between domains and mapping the features with domain invariance and distinguishability to obtain final abnormal labels; Wherein the preset longitudinal federal transfer model is constructed through a longitudinal federal feature extraction module, a domain adaptation module and a joint domain alignment module, and is obtained through training; Wherein the training is based on an unsupervised longitudinal federal transfer model pre-training process and a longitudinal federal transfer model training process.

2. The industry big data driven longitudinal federated migration anomaly detection method according to claim 1, wherein, In S2, the longitudinal federal feature extraction is based on a convolutional neural network and a spatial pyramid method to map the source domain dataset and the target domain dataset into a common feature space.

3. The industry big data driven longitudinal federated migration anomaly detection method of claim 2, wherein, In S2, the specific operation of longitudinal federal feature extraction is: By adding a spatial pyramid pooling layer before the full connection layer of the convolutional neural network, the spatial pyramid pooling layer includes adaptive convolution layers of different levels, the adaptive convolution automatically adjusts the shapes of the parameters of the convolution kernel, the padding and the stride by fixing the size of the output feature map, the shapes and sizes of the convolution kernel, the padding and the stride are back calculated through the following formulas (1), (2), (3), and the adaptive convolution layer forms potential features; kernel = H in - (H out - 1) * stride (2); Padding = 0 (3). wherein stride represents a stride, padding represents padding, and kernel represents a convolution kernel, denotes a floor function, H in denotes the size of the input data, H out denotes the feature output size.

4. The industry big data driven longitudinal federal migration anomaly detection method of claim 1, wherein, In S2, the domain adaptation is based on the principle of conditional domain adversarial learning to reduce the difference between different domain feature distributions, and extracts features with domain invariance and distinguishability from the latent features.

5. The industry big data driven longitudinal federated migration anomaly detection method of claim 4, wherein, In S2, the specific operation of domain adaptation is: By optimizing the latent feature alignment loss and the abnormal label classification loss of the source domain dataset based on the principle of adversarial learning, on the one hand, the objective function of the latent feature alignment loss is as follows: where E denotes expectation, f S and g S represent latent features of the source domain and classifier results output by the latent features, respectively, f T and g T correspond to latent features in the target domain and classifier results output by the latent features, respectively; Denotes a multilinear mapping; A sample weight method using the following equation (5) is used to avoid the influence of negative samples on training: Wherein p denotes the final prediction probability of each class by the classifier; optimizing the latent feature alignment loss based on an optimization method of joint domain adversarial learning; by minimizing the objective function of the latent feature alignment loss, the parameters of the feature extractor F and the classifier C are optimized, and by maximizing the objective function of the latent feature alignment loss, the parameters of the domain discriminator D are optimized, and the optimization process of each model parameter is as follows: where t denotes the tth iteration update, and Respectively denote the parameters of the feature extractor F, the classifier C and the domain discriminator D of the t-th training; On the other hand, the objective function of the abnormal label classification loss of the source domain dataset is as follows: wherein and respectively represent the i-th data sample and the corresponding label on the source domain; N S is the total number of samples of the source domain D S is the total number of samples of the source domain D S represents the feature extractor on the source domain.

6. The industry big data driven longitudinal federal migration anomaly detection method of claim 1, wherein, In S2, the joint domain alignment is based on a difference-based label distribution distance alignment method to map the features with domain invariance and distinguishability to final abnormal labels, realizing abnormal detection in the target domain without abnormal labels.

7. The industry big data driven longitudinal federated migration anomaly detection method of claim 6, wherein, In S2, the specific operation of joint domain alignment is: By adding the difference-based label distribution distance alignment method as the joint field alignment, the output label alignment loss is calculated, and the objective function of the joint field label alignment loss is as follows: wherein a latent feature representation representing the source domain input in the classification network at the output of the l-th layer, Denotes that each layer output of the classifier is projected to a Hilbert space through a multi-dimensional linear mapping, |L| denotes the number of layers in the classifier, and the last two layers of the classifier output are selected for alignment on different domains for the abnormal detection task.

8. The industry big data driven longitudinal federal migration anomaly detection method of claim 1, wherein, In S2, the unsupervised longitudinal federal transfer model pre-training process is used to reduce the training time and speed up the training convergence; specifically: The model input is the source domain dataset, the source domain feature extractor and the classifier parameters are randomly initialized, and then the objective function of the longitudinal federated transfer model pre-training loss is calculated, as shown in the following formula (10): Then the source domain feature extractor and classifier parameters are updated by backpropagation, as shown in equations (11) and (12): wherein μ is the learning rate; Denotes the gradient information of the loss objective function.

9. The industry big data driven longitudinal federal migration anomaly detection method of claim 1, wherein, In S2, the longitudinal federal transfer model training process trains the longitudinal federal transfer model based on a local and central server joint training method, specifically: The model input is a source domain data set and a target domain data set, the model parameter is initialized as the model parameter obtained after pre-training of the longitudinal federated transfer model, the source domain data set is input into the feature extractor and the classifier, the latent feature and the abnormal label classification loss are obtained, the latent feature and the gradient information are sent to the central server, the central server is waited to return the update information, and the parameters of the feature extractor and the classifier are updated through the following formula: where λ and β are hyperparameters; f S = F S (X S ) represents the extracted latent features on the source domain; Randomly select a target domain sample in the target domain, obtain the latent feature f passing through the feature extractor T and the objective function, send the latent feature to the central server, block and wait for the parameter update from the central server, and update the feature by the following formula Extractor and classifier parameters: Wherein μ denotes the learning rate, and t denotes the t-th iteration update; The central server receives all uploaded potential features, as well as the source domain corresponding labels, performs reverse labeling, and calculates the target function as follows: wherein anomaly label classification loss representing the source domain dataset; representative potential feature alignment loss; Represents the joint domain label alignment loss; The central server calculates the loss by forward propagation and obtains the gradient by back propagation, and transmits the gradient information to the source domain and the target domain.

10. An industry big data driven longitudinal federated migration anomaly detection system, comprising: Comprising a longitudinal federal feature extraction module, a domain adaptation module and a joint domain alignment module; wherein: The longitudinal federal feature extraction module is used to map the source domain dataset and the target domain dataset into a common feature space to obtain latent features; A domain adaptation module is configured to extract features with domain invariance and distinguishability from potential features. A joint domain alignment module is configured to align distances between domains and map the features with domain invariance and distinguishability to obtain final anomaly labels.

Images