Model training method, industrial internet anomaly detection method and related device

Abstract

The invention provides a model training method, an industrial internet anomaly detection method and a related device, and relates to the field of industrial internet. The model training method comprises the following steps: acquiring a training sample comprising a plurality of time sequence data samples; inputting the training sample into a domain projection module to perform feature extraction of a unified dimension to obtain a plurality of target features of a target dimension; inputting the plurality of target features into a feature mixing enhancement module for feature mixing enhancement processing to obtain positive features and negative features of a plurality of target dimensions; inputting the target features, the positive features and the negative features into an anomaly detection module for anomaly detection processing to obtain inter-block characterization and intra-block characterization corresponding to each feature, and characterization loss between the inter-block characterization and the intra-block characterization; and adjusting parameters of the anomaly detection model based on a first alignment loss among the plurality of target features, a second alignment loss among the plurality of in-block representations and a target loss determined by the representation loss, and obtaining the anomaly detection model with relatively good generalization ability.

Description

Technical Field

[0001] The present application relates to the field of industrial Internet, and in particular to a model training method, an industrial Internet anomaly detection method and related devices. Background Art

[0002] The Industrial Internet refers to a network system that deeply integrates Internet technology with industrial systems to improve industrial production efficiency, optimize resource allocation and promote innovation through sensors, communication technology, big data analysis and cloud computing.

[0003] At present, during the operation of the Industrial Internet, trained anomaly detection models can be used to detect anomalies in the Industrial Internet. Among them, the anomaly detection model is usually obtained by training based on machine learning or deep learning methods, but the generalization ability of the obtained anomaly detection model is poor. Summary of the invention

[0004] The embodiments of the present application provide a model training method, an industrial Internet anomaly detection method and related devices to solve the problem of poor generalization ability of the anomaly detection model obtained by current technology.

[0005] In a first aspect, an embodiment of the present application provides a model training method for training an anomaly detection model, wherein the anomaly detection model includes a domain projection module, a feature mixing enhancement module, an anomaly detection module, and a feature alignment module. The model training method includes: obtaining training samples, wherein the training samples include multiple time series data samples, and the multiple time series data samples are obtained by collecting operation data of different tasks when different tasks of the industrial Internet are in normal operation; inputting the multiple time series data samples into the domain projection module for feature extraction of unified dimensions, and obtaining target features of the target dimension corresponding to each time series data sample output by the domain projection module; inputting the multiple target features into the feature mixing enhancement module; The module performs feature mixing enhancement processing to obtain positive features and negative features of multiple target dimensions output by the feature mixing enhancement module; the target features, positive features and negative features are input into the anomaly detection module for anomaly detection processing to obtain inter-block representations and intra-block representations corresponding to the target features, positive features and negative features, respectively, as well as the representation loss between the inter-block representations and the intra-block representations; based on the first alignment loss between multiple target features, the second alignment loss between multiple intra-block representations and the representation loss, the target loss is determined, and the first alignment loss and the second alignment loss are obtained through the feature alignment module; the parameters of the anomaly detection model are adjusted according to the target loss to obtain a trained anomaly detection model.

[0006] Optionally, multiple target features are input into a feature mixing enhancement module for feature mixing enhancement processing to obtain positive features and negative features of multiple target dimensions output by the feature mixing enhancement module, including: performing pairwise summing processing on multiple target features to obtain positive features; and performing abnormal feature fragment insertion processing on multiple target features to obtain negative features.

[0007] Optionally, based on a first alignment loss between multiple target features, a second alignment loss between multiple intra-block representations, and a representation loss, a target loss is determined, including: inputting multiple target features into a feature alignment module for head alignment processing to obtain a first alignment loss; inputting multiple intra-block representations into a feature alignment module for tail alignment processing to obtain a second alignment loss; and determining the target loss as the sum of the first alignment loss, the second alignment loss, and the representation loss.

[0008] Optionally, the domain projection module includes a first projector and a second projector, and multiple time series data samples are input into the domain projection module for unified dimension feature extraction to obtain target features of the target dimension corresponding to each time series data sample output by the domain projection module, including: inputting the time series data samples into the first projector for dimensional unification processing and feature extraction to obtain initial features of the target dimension output by the first projector; inputting the initial features into the second projector for feature extraction to obtain target features output by the second projector.

[0009] Optionally, the target feature, positive feature and negative feature are input into an anomaly detection module for anomaly detection processing to obtain inter-block representation and intra-block representation corresponding to the target feature, positive feature and negative feature, respectively, as well as representation loss between the inter-block representation and the intra-block representation, including: for each feature of the target feature, positive feature and negative feature, the feature is processed in blocks to obtain block-processed features; based on the block-processed features, inter-block input features and intra-block input features are obtained; attention weight normalization and upsampling are performed on the inter-block input features and the intra-block input features to obtain inter-block representation and intra-block representation; and representation loss is obtained according to the difference between the inter-block representation and the intra-block representation.

[0010] In a second aspect, the present application provides an industrial Internet anomaly detection method, including: obtaining time series data to be detected, where the time series data to be detected is obtained by collecting the operating data of the target task of the industrial Internet; inputting the time series data to be detected into the domain projection module of the anomaly detection model to perform feature extraction of a unified dimension, and obtaining target features of the target dimension corresponding to the time series data to be detected output by the domain projection module, wherein the anomaly detection model is trained using the model training method described in the first aspect of the present application; inputting the target features into the anomaly detection module of the anomaly detection model to perform anomaly detection processing, and obtaining anomaly detection results corresponding to the time series data to be detected.

[0011] In a third aspect, the present application provides a model training device for training anomaly detection models, the anomaly detection model including a domain projection module, a feature mixing enhancement module, an anomaly detection module and a feature alignment module, the model training device including: an acquisition unit for acquiring training samples, the training samples including multiple time series data samples, the multiple time series data samples are obtained by collecting operation data of different tasks when different tasks of the industrial Internet are in normal operation; a feature extraction unit for inputting multiple time series data samples into the domain projection module for feature extraction of unified dimensions, and obtaining target features of the target dimension corresponding to each time series data sample output by the domain projection module; a feature mixing enhancement unit for inputting multiple target features into the feature mixing enhancement module; The strengthening module performs feature mixing enhancement processing to obtain positive features and negative features of multiple target dimensions output by the feature mixing enhancement module; the anomaly detection unit is used to input the target features, positive features and negative features into the anomaly detection module for anomaly detection processing to obtain inter-block representations and intra-block representations corresponding to the target features, positive features and negative features, respectively, and representation losses between the inter-block representations and the intra-block representations; the determination unit is used to determine the target loss based on a first alignment loss between multiple target features, a second alignment loss between multiple intra-block representations and the representation loss, wherein the first alignment loss and the second alignment loss are obtained through the feature alignment module; the adjustment unit is used to adjust the parameters of the anomaly detection model according to the target loss to obtain a trained anomaly detection model.

[0012] Optionally, the feature mixing enhancement unit is specifically used for: performing pairwise summing processing on multiple target features to obtain positive features; and performing insertion processing on abnormal feature fragments within the multiple target features to obtain negative features.

[0013] Optionally, the determination unit is specifically used to: input multiple target features into a feature alignment module for head alignment processing to obtain a first alignment loss; input multiple intra-block representations into a feature alignment module for tail alignment processing to obtain a second alignment loss; determine the target loss as the sum of the first alignment loss, the second alignment loss and the representation loss.

[0014] Optionally, the domain projection module includes a first projector and a second projector, and the feature extraction unit is specifically used to: input the time series data samples into the first projector for dimensional unification processing, and perform feature extraction to obtain the initial features of the target dimension output by the first projector; input the initial features into the second projector for feature extraction to obtain the target features output by the second projector.

[0015] Optionally, the anomaly detection unit is specifically used to: for each feature of the target feature, the positive feature and the negative feature, perform block processing on the feature to obtain block-processed features; based on the block-processed features, obtain inter-block input features and intra-block input features; perform attention weight normalization and upsampling on the inter-block input features and the intra-block input features to obtain inter-block representations and intra-block representations; and obtain representation loss based on the difference between the inter-block representations and the intra-block representations.

[0016] In a fourth aspect, the present application provides an industrial Internet anomaly detection device, comprising:

[0017] An acquisition unit is used to acquire the time series data to be detected, where the time series data to be detected is obtained by collecting the operation data of the target task of the industrial Internet;

[0018] A feature extraction unit, used to input the time series data to be detected into the domain projection module of the anomaly detection model to perform feature extraction of unified dimension, and obtain the target feature of the target dimension corresponding to the time series data to be detected output by the domain projection module, wherein the anomaly detection model is trained by using the model training method as described in the first aspect of the present application;

[0019] The anomaly detection unit is used to input the target features into the anomaly detection module of the anomaly detection model for anomaly detection processing to obtain anomaly detection results corresponding to the time series data to be detected.

[0020] In a fifth aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0021] Memory stores computer-executable instructions;

[0022] The processor executes computer execution instructions stored in the memory to implement the model training method described in the first aspect of this application or the industrial Internet anomaly detection method described in the second aspect.

[0023] In a sixth aspect, the present application provides a computer-readable storage medium, which stores computer program instructions. When the computer program instructions are executed, the model training method described in the first aspect of the present application or the industrial Internet anomaly detection method described in the second aspect is implemented.

[0024] In the seventh aspect, the present application provides a computer program product, including a computer program, which, when executed, implements the model training method described in the first aspect of the present application or the industrial Internet anomaly detection method described in the second aspect.

[0025] The present application provides a model training method, an industrial Internet anomaly detection method and related devices, wherein the anomaly detection model includes a domain projection module, a feature mixing enhancement module, an anomaly detection module and a feature alignment module; by obtaining training samples, the training samples include multiple time series data samples, and the multiple time series data samples are input into the domain projection module for feature extraction of unified dimensions, so as to obtain target features of the target dimension corresponding to each time series data sample output by the domain projection module, so as to facilitate better data analysis and training of the anomaly detection model; multiple target features are input into the feature mixing enhancement module for feature mixing enhancement processing, so as to obtain positive features and negative features of multiple target dimensions output by the feature mixing enhancement module, which can increase the number and diversity of training features and solve the problem of scarcity of abnormal data, thereby improving the anomaly detection model's ability to recognize anomalies; the target features, positive features and negative features are input into the anomaly detection module for anomaly detection processing, so as to obtain The target features, positive features and negative features correspond to the inter-block representation and intra-block representation respectively, as well as the representation loss between the inter-block representation and the intra-block representation. The difference between the inter-block representation and the intra-block representation can be reduced through the representation loss. The target loss is determined based on the first alignment loss between multiple target features, the second alignment loss between multiple intra-block representations and the representation loss. The first alignment loss and the second alignment loss are obtained through the feature alignment module. By obtaining the first alignment loss and the second alignment loss, the problem of feature distribution mismatch between different tasks can be solved, the noise in the training process can be reduced, and the knowledge transfer between tasks can be ensured to be more accurate and effective, and the generalization of the anomaly detection model can be enhanced. Therefore, the parameters of the anomaly detection model are adjusted according to the target loss to obtain the trained anomaly detection model. The obtained anomaly detection model has good generalization ability and can more accurately detect anomalies in the time series data corresponding to different tasks in the industrial Internet. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0027] Figure 1 It is a schematic diagram of single-task learning provided by related technologies;

[0028] Figure 2 It is a schematic diagram of transfer learning provided by related technologies;

[0029] Figure 3 It is a schematic diagram of multi-task learning provided by related technologies;

[0030] Figure 4 A flowchart of a model training method provided in one embodiment of the present application;

[0031] Figure 5A flowchart of a model training method provided in another embodiment of the present application;

[0032] Figure 6 A schematic diagram of a specific training process of a model provided in an embodiment of the present application;

[0033] Figure 7 A flowchart of an industrial Internet anomaly detection method provided in one embodiment of the present application;

[0034] Figure 8 A schematic diagram of the structure of a model training device provided in one embodiment of the present application;

[0035] Fig. 9 A schematic diagram of the structure of an industrial Internet anomaly detection device provided in one embodiment of the present application;

[0036] Fig.10 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application.

[0037] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0039] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0040] First, some technical terms involved in this application are explained:

[0041] One-Class Learning: During the learning process, the model is only exposed to data of one category (class label) and no data of any other category.

[0042] Transfer Learning (TL): During the learning process, the model acquires knowledge from the source domain and transfers it to the target domain for use;

[0043] Multi-Task Learning (MTL), in which the model is trained on multiple interrelated tasks simultaneously during the learning process, with the ultimate goal of improving the generalization ability between different tasks and the performance on each task;

[0044] Domain projection is the process of transforming two sets of data of different dimensions from the original feature space to a common dimensional space to facilitate better data analysis and model training;

[0045] Feature mixture enhancement is a technique that improves the performance and generalization ability of the model by increasing the diversity and amount of training data;

[0046] Feature alignment: For normal data from different data domains, through training, the model can extract features with similar distribution in the feature space.

[0047] At present, during the operation of the Industrial Internet, trained anomaly detection models can be used to detect anomalies in the Industrial Internet. Among them, the anomaly detection model is usually trained based on machine learning or deep learning methods. Specifically, the methods based on machine learning and deep learning mainly include two categories: supervised learning methods and unsupervised learning methods. Among them, supervised learning methods require a large amount of high-quality labeled normal data and abnormal data. However, labeled data and abnormal data are relatively scarce, which limits the application of supervised learning methods in anomaly detection. Among them, the main reasons for the scarcity of labeled data and abnormal data include: (1) The scarcity of abnormal events: In industrial scenarios, abnormal events usually occur at a low frequency, so it is difficult to collect enough abnormal samples; (2) High labeling cost: Obtaining high-quality labeled data requires professional knowledge, and the labeling process is time-consuming and labor-intensive, resulting in high labeling costs; (3) Privacy and security issues: In some sensitive fields, such as medical or financial fields, data sharing and labeling may be restricted by privacy protection and security regulations; (4) Dynamic environment: The industrial environment is dynamically changing, and abnormal patterns may change over time, making it difficult for historical data to reflect the current situation. Therefore, anomaly detection models mainly apply unsupervised learning methods.

[0048] Unsupervised learning methods usually learn the normal operating mode of the system from normal data to identify anomalies during the reasoning process. Unsupervised learning methods can also be called single-class learning methods. Specifically, in a related technology, the OmniAnomaly model (a deep learning model for time series anomaly detection) can be used for anomaly detection, and the normal mode of multivariate time series data can be captured by learning key technologies including random variable connection and plane normalization flow. Among them, the generative adversarial network is incorporated into the OmniAnomaly model, so that more stable detection results can be obtained. In another related technology, Anomaly Transformer (a deep learning model for anomaly detection) can also be used for anomaly detection, in which a novel anomaly attention mechanism is introduced to obtain correlation differences, and the minimum and maximum strategies are used to improve the distinguishability between normal and abnormal patterns based on correlation differences. In another related technology, DCdetector (a multi-scale dual attention contrast representation learning model) can also be used for anomaly detection, which combines asymmetric dual attention to construct an arrangement environment to obtain arrangement invariant representation. Although unsupervised learning methods do not require a large amount of labeled abnormal data, they rely on learning patterns from normal data and may not be able to identify new anomalies that are very different from the training data. Therefore, the generalization ability of anomaly detection models trained based on machine learning or deep learning methods is poor.

[0049] In addition, current methods focus on transfer learning (TL) to solve the problem of data scarcity in the field of anomaly detection. Transfer learning enables the knowledge of a model trained from source domain data to be transferred to a related but different target domain, thereby reducing model bias and improving prediction accuracy. However, current anomaly detection migration techniques still face challenges. First, methods based on transfer learning are not capable of eliminating differences between data in different domains. This method usually only focuses on the initial unification of data features and ignores the subsequent unification of features from different tasks. Second, the comprehensive utilization of data is insufficient. This method usually only transfers knowledge from the source domain to the target domain without effectively utilizing the source domain data itself. In addition, since the goal of transfer learning-based methods is to improve the performance of the target domain, the resulting models often fail to achieve excellent results on the source domain dataset, making this method unsuitable for processing cross-domain tasks. It can be understood that transfer learning aims to transfer knowledge from the source domain to the target domain, but in anomaly detection, differences between domains may lead to poor migration effects. Existing methods based on transfer learning are insufficient in eliminating domain differences, especially in the subsequent unification of feature space. Existing transfer learning-based methods usually only focus on the knowledge transfer from the source domain to the target domain, without fully considering how to use the source domain data itself to enhance the anomaly detection capability of the target domain. The main reasons for the domain adaptability problem of transfer learning include: (1) Distribution difference: There is a difference in the data distribution of the source domain and the target domain, which may lead to a decrease in the performance of the model on the target domain; (2) Feature mismatch: The feature spaces of the source domain and the target domain may be inconsistent, making feature mapping and adaptation in transfer learning complicated; (3) Lack of domain adaptation technology: Existing transfer learning-based methods may lack effective domain adaptation technology and cannot well adjust the model to adapt to the new data distribution; (4) Ignoring domain-specific information: In the process of transfer learning, transfer learning-based methods may ignore the information specific to the target domain, resulting in poor transfer effect.

[0050] Currently, cross-domain anomaly detection can also be performed based on multi-task learning (MTL) methods. Multi-task learning (MTL) represents a paradigm that trains multiple interrelated tasks simultaneously, thereby promoting knowledge transfer across tasks, with the ultimate goal of improving the generalization ability of each considered task. Unlike single task learning (STL) and TL, MTL simulates data augmentation by pooling labeled data across tasks to improve learner accuracy in all tasks. TL aims to improve the performance of the target task with the help of the source task. Exemplarily, Figure 1 It is a schematic diagram of single-task learning provided by related technologies. Figure 2 It is a schematic diagram of transfer learning provided by related technologies. Figure 3 It is a schematic diagram of multi-task learning provided by related technologies. Figure 1 , Figure 2 and Figure 3 In comparison, for single-task learning, taking Task 1 from Task 1 to Task n as an example, the training data of Task 1 is used to train the model, and the trained model is used to detect anomalies on the data of Task 1; for transfer learning, the training data corresponding to Task 1 to Task n are input into the model in sequence for training, and the trained model is used to detect anomalies on the data of the target task; for multi-task learning, the training data corresponding to Task 1 to Task n are input into the model together for training, and the trained model is used to detect anomalies on the data corresponding to Task 1 to Task n. It can be understood that MTL can promote knowledge reuse and reduce the burden of labels. The effectiveness of MTL stems from its ability to utilize diverse data from multiple tasks to obtain more robust and more general representations, which will also improve the performance of all tasks, improve knowledge transfer, and reduce the risk of overfitting.

[0051] Specifically, the methods of multi-task learning (MTL) for cross-domain anomaly detection mainly include: (1) a self-supervised method can be adopted, in which a three-dimensional (3D) convolutional neural network (backbone network) is trained to generate discriminative anomaly-specific information by simultaneously learning multiple proxy tasks, and the 3D convolutional backbone network is enhanced by integrating a multi-head self-attention module; (2) a self-supervised learning framework can be adopted for deep anomaly detection, which can address the limitations of feature granularity and task specificity; (3) a unified autoencoder (UAE) can be used to initially extract a shared representation of input weather features across tasks, and then a task embedding layer is applied in a neural network (TENN) to obtain task-specific information; (4) a modular framework can be designed for extracting semantic embeddings and then trained through deep metric learning; (5) an innovative method can be adopted to learn task-shared feature representations and transfer anomaly detection rules from offline normal data to online unlabeled data; (6) a multi-task learning (MTL) framework can be introduced, which integrates long short-term memory (LSTM) with deep neural network (TENN). However, under the MTL framework, research on time series anomaly detection is still quite scarce.

[0052] Understandably, despite the advantages of multi-task learning in promoting cross-task knowledge transfer and improving model generalization ability, there is relatively little research on time series anomaly detection. Multi-task learning methods may not be mature enough to deal with the dynamic changes and complexity of time series data, and it is difficult to effectively capture abnormal patterns in time series data. In addition, the design and optimization of multi-task learning frameworks are usually complex, requiring a large amount of task-specific data and fine-tuning of models, which may be difficult to achieve in practical applications. The main reasons for the application limitations of multi-task learning in time series anomaly detection include: (1) Model complexity: The model that needs to optimize multiple time series data anomaly detection tasks at the same time is usually more complex than the model for a single task; (2) Task correlation: Multi-task learning assumes that there is correlation between tasks, but in time series anomaly detection, the correlation between different tasks may not be obvious or difficult to model; (3) Resource allocation: In multi-task learning, how to reasonably allocate resources to optimize multiple tasks at the same time is a difficult problem, especially when resources are limited; (4) Time series characteristics: Time series data has complex characteristics such as serial dependence, seasonality, and trend, which makes it difficult for multi-task learning models to capture and utilize these characteristics for effective anomaly detection; (5) Insufficient research attention: Compared with single-task learning and transfer learning, there is relatively little research on multi-task learning in the field of time series anomaly detection, resulting in a low maturity of related theories and methods.

[0053] Based on the above problems, the present application provides a model training method, an industrial Internet anomaly detection method and related devices. When training the anomaly detection model, based on a multi-task learning method, by collecting operating data of different tasks of the industrial Internet in normal operating state as time series data samples, the time series data samples corresponding to different tasks are input into the anomaly detection model for training; wherein, a unified dimension feature extraction is performed on the time series data samples to obtain a target feature of the target dimension corresponding to each time series data sample; positive features and negative features are constructed based on the target features, which can increase the number and diversity of training features and improve the anomaly recognition ability of the anomaly detection model; and then the inter-block representation and intra-block representation corresponding to the target features, positive features and negative features are obtained, respectively, as well as the representation loss between the inter-block representation and the intra-block representation; based on a first alignment loss between multiple target features, a second alignment loss between multiple intra-block representations and the representation loss, the target loss is determined. By obtaining the first alignment loss and the second alignment loss, the problem of feature distribution mismatch between different tasks can be solved, ensuring that the knowledge transfer between tasks is more accurate and effective; thereby adjusting the parameters of the anomaly detection model according to the target loss to obtain a trained anomaly detection model. The obtained anomaly detection model has good generalization ability and can more accurately detect anomalies in the operating data of different tasks of the industrial Internet.

[0054] It should be noted that the model training method and industrial Internet anomaly detection method provided in the embodiments of the present application can be applied in a server, which can be an independent server, or a service cluster, etc.

[0055] Figure 4 The flowchart of the model training method provided in one embodiment of the present application is used to train an anomaly detection model, the anomaly detection model includes a domain projection module, a feature mixing enhancement module, an anomaly detection module and a feature alignment module. Figure 4 As shown, the method of the embodiment of the present application includes:

[0056] S401. Obtain training samples, where the training samples include multiple time series data samples. The multiple time series data samples are obtained by collecting operating data of different tasks when different tasks of the industrial Internet are in normal operating state.

[0057] In the embodiment of the present application, the industrial Internet includes multiple production lines, and one production line can be understood as a task. The operating data of different tasks in normal operation can be collected as samples at preset time intervals (for example, every 1 second), so that multiple time series data samples corresponding to different tasks can be obtained. Among them, multiple time series data samples can be understood as positive samples, and multiple time series data samples are data under the same time window. The embodiment of the present application does not limit the time window of the time series data sample, and the dimensions of multiple time series data samples may be different. When collecting data, for example, data collection can be performed through sensors, or data collection can be performed through a supervisory control and data acquisition (SCADA) system.

[0058] S402 , inputting multiple time series data samples into a domain projection module to extract features of a unified dimension, and obtaining target features of a target dimension corresponding to each time series data sample output by the domain projection module.

[0059] It can be understood that the domain projection module aims to extract features of unified dimension and similar distribution across domains (across tasks). The domain projection module is implemented based on a multilayer perceptron (MLPs) model, for example, and can also be implemented using, for example, a convolutional neural network (CNN), a recurrent neural network (RNN), a long short-term memory network, or an attention network. Exemplarily, multiple (specifically, for example, 2) time series data samples are input into the domain projection module, and the domain projection module independently encodes the time series data samples from different tasks to obtain the encoded features corresponding to each time series data sample, and projects the encoded features into a shared feature space to obtain the target features of the target dimension corresponding to each time series data sample, and the dimensions of the multiple target features are the same. Specifically, the input of the domain projection module is a time series data sample from any task (for example, with ), then , where T represents the data window length hyperparameter of the time series data sample, c represents the dimension of the input data (time series data sample), and c of time series data samples of different tasks is generally different. represents the original feature space; the output of the domain projection module is the target feature of the target dimension in the shared feature space (for example, ), then , with the same number of channels d and unchanged data length, represents the shared feature space.

[0060] S403, inputting multiple target features into a feature mixing enhancement module for feature mixing enhancement processing, and obtaining positive features and negative features of multiple target dimensions output by the feature mixing enhancement module.

[0061] It can be understood that the feature mixing enhancement module is intended to increase the number and diversity of training features. The input of the feature mixing enhancement module is the different target features extracted by the domain projection module, and the output is the constructed mixed features. Among them, according to the construction method adopted, the mixed features are defined as positive features or negative features. Positive features are intended to enhance the attention of the anomaly detection model to the common features of cross-domain data, while negative features are intended to enhance the sensitivity of the anomaly detection model to abnormal patterns. Exemplarily, multiple target features are input into the feature mixing enhancement module, for example, any two target features among the multiple target features can be summed to obtain positive features of the target dimension; any one of the multiple target features can be processed by inserting abnormal feature fragments within the feature to obtain negative features of the target dimension, wherein the abnormal feature fragment within the feature is, for example, a feature fragment obtained from any target feature other than the current target feature.

[0062] S404: Input the target features, positive features and negative features into the anomaly detection module for anomaly detection processing to obtain the inter-block representation and intra-block representation corresponding to the target features, positive features and negative features, respectively, and the representation loss between the inter-block representation and the intra-block representation.

[0063] It can be understood that the anomaly detection module can adapt to many unsupervised time series anomaly detection models. Considering the flexibility in integrating diverse existing models and the ability to improve anomaly detection performance, the anomaly detection module in the embodiment of the present application can be implemented, for example, using the backbone network of the time series anomaly detection model DCdetector or Anomaly Transformer, and can also be implemented by, for example, PatchAD (an anomaly detection network). The backbone networks of DCdetector and AnomalyTransformer are both dual-branch structures designed based on contrastive learning. In this step, after obtaining the target features, positive features, and negative features, the target features, positive features, and negative features can be input into the anomaly detection module for anomaly detection processing, thereby obtaining the inter-block representation and intra-block representation corresponding to the target features, positive features, and negative features, respectively, and the representation loss between each group of inter-block representations and intra-block representations. For how to obtain the inter-block representation and intra-block representation corresponding to the target features, positive features, and negative features, respectively, and the representation loss between the inter-block representation and the intra-block representation, please refer to the subsequent embodiments, which will not be repeated here.

[0064] S405, determining a target loss based on a first alignment loss between multiple target features, a second alignment loss between multiple intra-block representations, and a representation loss, wherein the first alignment loss and the second alignment loss are obtained through a feature alignment module.

[0065] It can be understood that the feature alignment module is intended to enable the anomaly detection model to extract similar cross-domain (cross-task) features as much as possible. Similar cross-domain features represent the intrinsic properties of multi-dimensional time series data in different fields (tasks). Exemplarily, multiple target features can be input into the feature alignment module, and head alignment processing is performed based on the adversarial training method to obtain a first alignment loss; multiple intra-block representations are input into the feature alignment module, and tail alignment processing is performed based on the adversarial training method to obtain a second alignment loss, so that the target loss can be determined based on the first alignment loss between multiple target features, the second alignment loss between multiple intra-block representations, and the representation loss. The target loss is, for example, the sum of the first alignment loss, the second alignment loss, and the representation loss.

[0066] S406: Adjust the parameters of the anomaly detection model according to the target loss to obtain a trained anomaly detection model.

[0067] In this step, after the target loss is determined, the parameters of the anomaly detection model can be adjusted according to the target loss to obtain a trained anomaly detection model.

[0068] It can be understood that the anomaly detection model includes four key modules: domain projection module, feature mixing enhancement module, anomaly detection module and feature alignment module. Among them, the domain projection module is used to map the time series data of different domains (tasks) to a shared feature space of the same dimension; the feature mixing enhancement module can fuse the features of different domains to construct new features (positive features or negative features); the anomaly detection module can generate representation features and anomaly detection results based on the existing anomaly detection network; the feature alignment module includes head alignment and tail alignment. The head alignment is used to process the target features output by the domain projection module, while the tail alignment is used to process the representation features (i.e., intra-block representation) output by the anomaly detection module.

[0069] The model training method provided in the embodiment of the present application obtains training samples, and the training samples include multiple time series data samples. The multiple time series data samples are input into the domain projection module to extract features of a unified dimension, and the target features of the target dimension corresponding to each time series data sample output by the domain projection module are obtained, so as to facilitate better data analysis and training of the anomaly detection model; the multiple target features are input into the feature mixing enhancement module to perform feature mixing enhancement processing, and the positive features and negative features of the multiple target dimensions output by the feature mixing enhancement module are obtained, which can increase the number and diversity of training features and solve the problem of scarcity of abnormal data, thereby improving the anomaly recognition ability of the anomaly detection model; the target features, positive features and negative features are input into the anomaly detection module to perform anomaly detection processing, and the target features, positive features and negative features are obtained. The inter-block representation and intra-block representation corresponding to the features, as well as the representation loss between the inter-block representation and the intra-block representation, can reduce the difference between the inter-block representation and the intra-block representation through the representation loss; based on the first alignment loss between multiple target features, the second alignment loss between multiple intra-block representations and the representation loss, the target loss is determined, the first alignment loss and the second alignment loss are obtained through the feature alignment module, and by obtaining the first alignment loss and the second alignment loss, the problem of feature distribution mismatch between different tasks can be solved, the noise in the training process can be reduced, the knowledge transfer between tasks can be ensured to be more accurate and effective, and the generalization of the anomaly detection model can be enhanced; thereby, the parameters of the anomaly detection model are adjusted according to the target loss to obtain a trained anomaly detection model, and the obtained anomaly detection model has good generalization ability.

[0070] Figure 5 This is a flow chart of a model training method provided by another embodiment of the present application. Based on the above embodiment, the present application embodiment further illustrates the model training method. Figure 5 As shown, the method of the embodiment of the present application may include:

[0071] S501. Obtain training samples, where the training samples include multiple time series data samples. The multiple time series data samples are obtained by collecting operating data of different tasks when different tasks of the industrial Internet are in normal operating state.

[0072] The detailed description of this step can be found in Figure 4 The relevant description of S401 in the illustrated embodiment will not be repeated here.

[0073] Considering that the domain projection module includes a first projector and a second projector, therefore, in the embodiment of the present application, Figure 4 The step S402 may further include the following two steps S502 and S503:

[0074] S502: Input the time series data samples into the first projector for dimension unification processing and feature extraction to obtain initial features of the target dimension output by the first projector.

[0075] S503: Input the initial features into the second projector for feature extraction to obtain target features output by the second projector.

[0076] It can be understood that initially, the domain projection module independently encodes the time series data samples from different tasks to obtain the encoding features corresponding to the time series data samples of different tasks. Subsequently, the domain projection module projects the encoded features into a shared feature space. Among them, the domain projection module includes two key components, namely the first projector and the second projector. The first projector can also be understood as a private projector, and the second projector can also be understood as a public projector. The first projector and the second projector are both implemented based on the MLPs model, and can also be implemented by, for example, a convolutional neural network, a recurrent neural network, a long short-term memory network or an attention network. Among them, the first projector (private projector) independently processes the time series data samples from different tasks to achieve dimensional unification and initial feature extraction of the target dimension; the second projector (public projector) receives the initial features of the target dimension output by each private projector to further extract the target feature. If the first projector and the second projector are both implemented based on the MLPs model, the formula of the multilayer perceptron is expressed as the following formula 1:

[0077]

[0078] in, , Indicates parameters of hidden layers; " represents matrix multiplication; BatchNorm is the batch normalization function, and ReLU is the nonlinear activation function; Represents input data (e.g., time series data samples); Represents the parameters of the hidden layer.

[0079] For example, Figure 6 This is a schematic diagram of a specific model training process provided in an embodiment of the present application. Figure 6 As shown, the time series data samples corresponding to any two tasks (i.e., task i and task j) in multiple time series data samples are and For example, and The dimensions are different; and The input domain projection module performs unified dimension feature extraction, and the domain projection module outputs The target characteristics of the corresponding target dimension as well as The target characteristics of the corresponding target dimension .

[0080] In the embodiment of the present application, Figure 4 The step S403 may further include the following two steps S504 and S505:

[0081] S504: Input multiple target features into a feature mixing enhancement module, perform summation processing on multiple target features in pairs, and obtain positive features of the target dimension.

[0082] S505. Insert abnormal feature fragments into multiple target features through a feature mixing enhancement module to obtain negative features of the target dimension.

[0083] For example, reference Figure 6 ,definition , is a random number. By summing up multiple target features in pairs, that is, constructing positive features based on the addition of each time point, the positive features of the target dimension are expressed as the following formula 2:

[0084]

[0085] in, represents the positive features of the construct; and It is the target feature from the output of the domain projection module.

[0086] The feature mixing enhancement module is used to insert abnormal feature fragments into multiple target features, that is, to construct negative features based on slice splicing, and the negative features of the target dimension are obtained through the following formula 3:

[0087]

[0088] in, represents the negative features of the construct; represents the concatenation function along the time dimension; and Indicates the random starting and ending positions of the inserted abnormal feature fragment; Indicates that the insertion The abnormal characteristic fragments are shown in Figure 2, and the rest of the fragments are normal fragments.

[0089] It should be noted that the embodiment of the present application does not limit the execution order of S504 and S505.

[0090] In the embodiment of the present application, Figure 4 The step S404 may further include the following step S506:

[0091] S506. For each feature of the target feature, positive feature and negative feature, the feature is processed in blocks through the anomaly detection module to obtain block-processed features; based on the block-processed features, inter-block input features and intra-block input features are obtained; attention weight normalization and upsampling are performed on the inter-block input features and intra-block input features to obtain inter-block representations and intra-block representations corresponding to the target feature, positive feature and negative feature, respectively; representation loss is obtained according to the difference between the inter-block representation and the intra-block representation.

[0092] For example, taking the anomaly detection module using the DCdetector backbone network as an example, referring to the example of step S402, the backbone network first performs an input target feature Perform block processing to obtain the features after block processing , where p represents the block size and s represents the number of blocks. Then we take the average along the dimensions of p and s to get the inter-block input features and the input features within the block Finally, according to the following formula 4 and formula 5, we can obtain the inter-block representation (for example, represented by P) and the intra-block representation (for example, represented by S), is the output of the backbone network.

[0093]

[0094] in, are the query (Q) and key (K) matrices used for attention calculation, obtained by matrix multiplication of the input and weight matrices, e.g., , They represent the corresponding weight matrices respectively; the Softmax function is used to normalize the attention weights; the Upsample function represents the upsampling function, which is used to convert S and P to the same length.

[0095] During the training process, DCdetector can define the representation loss of each branch according to the difference between the inter-block representation and the intra-block representation, such as the relative entropy KL divergence, through the following formulas 6 and 7:

[0096]

[0097] in, Indicates the representation loss corresponding to the branch where P is located, The StopGrad() function indicates the representation loss corresponding to the branch where S is located; the StopGrad() function indicates the truncation of the gradient propagation path. The representation loss corresponding to the time series data sample (i.e., the target feature) is obtained through the following formula 8:

[0098]

[0099] By optimizing the representation loss shown in Formula 8, the difference between the inter-block representation and the intra-block representation can be reduced. Figure 6 , the inter-block representation, intra-block representation and representation loss corresponding to task i are defined as ; The inter-block representation, intra-block representation and representation loss corresponding to task j are defined as .

[0100] For the constructed positive and negative features, the inter-block representation and intra-block representation corresponding to the positive features can be used respectively and The inter-block representation and intra-block representation corresponding to the negative features are represented by and For the constructed positive features, the representation loss corresponding to the positive features is obtained by the following formula 9:

[0101]

[0102] For the constructed negative features, the representation loss corresponding to the negative features is obtained by the following formula 10:

[0103]

[0104] It should be noted that the Calculate the interval for the inserted abnormal feature fragment , mainly used to emphasize the anomalous nature of the inserted anomalous feature fragment compared to the complete fragment.

[0105] During the model training process, different forms of loss functions can be used, such as loss functions based on cross entropy loss, JS divergence (a symmetric and normalized measure based on KL divergence) or mean square error. It can be understood that the positive features constructed by adding each time point can be regarded as the superposition of two unrelated features in the time domain, and the information of the original features is maintained in the frequency domain. Therefore, they can be used as positive features (positive samples); in addition, the role of the loss function for positive features in the training of anomaly detection models is to enhance the similarity between intra-block representation and inter-block representation; based on the negative features constructed by slice splicing, the abnormal feature fragments inserted in the middle obscure the original time series change process in the time domain, and the information contained in the frequency domain is completely different from the other fragments. Therefore, the feature mixing enhancement result can be used as a negative feature (negative sample); in addition, the role of the loss function for negative features in the training of anomaly detection models is to enhance the sensitivity of the anomaly detection model to abnormal fragments.

[0106] In the embodiment of the present application, Figure 4 The step S405 may further include the following three steps S507 to S509:

[0107] S507 , inputting the multiple target features into a feature alignment module for head alignment processing to obtain a first alignment loss.

[0108] S508 , performing tail alignment processing on the input feature alignment modules of the multiple intra-block representations to obtain a second alignment loss.

[0109] For example, reference Figure 6 The input of the feature alignment module is the classifier of the anomaly detection model (used to distinguish the sources of the two input time series data samples, so that and The distribution of is closer to the two intermediate features (i.e. and ), the output is the loss of the anomaly detection model and the discriminator of the anomaly detection model. For feature alignment, the loss functions of the adversarial training of the anomaly detection model and the discriminator are defined as the following formula 11 and formula 12 respectively:

[0110]

[0111] Among them, when the adversarial training loss of the anomaly detection model is back-propagated, only the model parameter gradients are recorded and the discriminator parameter gradients are discarded. When the discriminator loss is back-propagated, only the gradients on the discriminator parameters are recorded. The discriminator is optimized separately, while the adversarial loss of the anomaly detection model is optimized simultaneously with other loss functions. Optionally, feature alignment can also be performed based on other feature alignment methods, such as feature alignment based on the Maximum Mean Discrepancy (MMD) loss function.

[0112] For example, reference Figure 6 Based on the above formula 11 and formula 12, the anomaly detection model applies feature alignment twice. The first feature alignment acts on and , called head alignment, the first alignment loss is obtained by Indicates that, at the same time, the first discriminator loss can also be obtained, such as using The second feature alignment acts on the intra-block representation and , called tail alignment, the second alignment loss is obtained by Indicates that, at the same time, the second discriminator loss can also be obtained, such as using It can be understood that the embodiment of the present application applies two feature alignments, namely head alignment and tail alignment, and adopts adversarial learning method to achieve feature alignment, which can enhance the consistency of the features of data from different domains extracted by the anomaly detection model, reduce the noise in the training process, and enhance the generalization of the anomaly detection model; using head alignment and tail alignment at the same time, the input features and output features of cross-domain data can be aligned respectively, and the stability of data processing of the anomaly detection model can be enhanced.

[0113] It should be noted that the embodiment of the present application does not limit the execution order of S507 and S508.

[0114] S509: Determine the target loss as the sum of the first alignment loss, the second alignment loss, and the characterization loss.

[0115] Exemplarily, the final target loss is determined by the following formula 13 (for example, using express):

[0116]

[0117] The meaning of each parameter may refer to the above embodiments.

[0118] S510: Adjust the parameters of the anomaly detection model according to the target loss to obtain a trained anomaly detection model.

[0119] The detailed description of this step can be found in Figure 4 The relevant description of S406 in the illustrated embodiment will not be repeated here.

[0120] After obtaining the trained anomaly detection model, in the testing phase of the anomaly detection model, it can be conducted on the test set of any task corresponding to the training set (i.e., training samples). For example, a score-based anomaly detection processing method can be applied, and the anomaly score (AS) is defined as the following formula 14:

[0121]

[0122] in, and They represent the inter-block representation and intra-block representation corresponding to the input data of the test set respectively; AS is used to reflect the difference between the outputs of two branches. The greater the difference between the two branches, the more likely the corresponding position is an outlier.

[0123] Based on the anomaly score and the preset hyperparameter anomaly threshold (such as ), defining the anomaly detection results at each time point (for example, using The formula is as follows:

[0124]

[0125] Among them, if AS is greater than the preset hyperparameter abnormal threshold , then the output 1 (indicates that the data is abnormal at a certain point in time); if AS is less than or equal to the preset hyperparameter abnormal threshold , then the output is 0 (indicating that the data is normal at a certain time point). By obtaining the KL divergence between the two branch distributions, a quantitative anomaly score can be obtained to determine whether the data at each time point in the time series data in the test set is abnormal.

[0126] The model training method provided in the embodiment of the present application obtains training samples, and the training samples include multiple time series data samples. The first projector is used to unify the dimensions of the time series data samples and extract initial features, and the second projector is used to further extract target features, so as to facilitate better data analysis and training of anomaly detection models; a feature mixing enhancement module is used to perform a summation process between two of the multiple target features to obtain positive features of the target dimension, and an abnormal feature fragment is inserted into the multiple target features to obtain negative features of the target dimension, which can increase the number and diversity of training features and solve the problem of scarcity of abnormal data, thereby improving the anomaly recognition ability of the anomaly detection model; for each feature of the target feature, the positive feature and the negative feature, the block table corresponding to the target feature, the positive feature and the negative feature is obtained through the anomaly detection module. The difference between the inter-block representation and the intra-block representation is obtained according to the representation loss. The difference between the inter-block representation and the intra-block representation can be reduced by the representation loss. Multiple target features are input into the feature alignment module for head alignment to obtain the first alignment loss. Multiple intra-block representations are input into the feature alignment module for tail alignment to obtain the second alignment loss. By obtaining the first alignment loss and the second alignment loss, the problem of feature distribution mismatch between different tasks can be solved, the noise in the training process can be reduced, the knowledge transfer between tasks can be ensured to be more accurate and effective, and the generalization of the anomaly detection model can be enhanced. Therefore, the target loss is determined to be the sum of the first alignment loss, the second alignment loss and the representation loss. Then, the parameters of the anomaly detection model can be adjusted according to the target loss to obtain the trained anomaly detection model. The obtained anomaly detection model has good generalization ability.

[0127] Based on the above embodiments, Figure 7 A flowchart of an industrial Internet anomaly detection method provided in one embodiment of the present application. Figure 7 As shown, the method of the embodiment of the present application includes:

[0128] S701. Obtain the time series data to be tested. The time series data to be tested is obtained by collecting the operating data of the target task of the industrial Internet.

[0129] In this step, the operating data of the target task of the Industrial Internet can be collected to obtain the time series data to be detected. The embodiment of the present application does not limit the time window of the time series data to be detected. The target task can be understood as any task (production line) in the Industrial Internet, not a specific task in the Industrial Internet.

[0130] S702: Input the time series data to be detected into the domain projection module of the anomaly detection model to obtain the target features of the target dimension corresponding to the time series data to be detected output by the domain projection module. The anomaly detection model is trained using the model training method in any of the above method embodiments.

[0131] In this step, the anomaly detection model includes a domain projection module, a feature mixing enhancement module, an anomaly detection module, and a feature alignment module. When applying the anomaly detection model, only the domain projection module and the anomaly detection module of the anomaly detection model are needed. Therefore, the time series data to be detected can be input into the domain projection module of the anomaly detection model to extract features of a unified dimension, and the target features of the target dimension corresponding to the time series data to be detected output by the domain projection module can be obtained.

[0132] S703: Input the target feature into the anomaly detection module of the anomaly detection model for anomaly detection processing to obtain anomaly detection results corresponding to the time series data to be detected.

[0133] In this step, after obtaining the target features of the target dimension corresponding to the time series data to be detected, the target features can be input into the anomaly detection module of the anomaly detection model for anomaly detection processing to obtain anomaly detection results corresponding to the time series data to be detected. When the anomaly detection module performs anomaly detection processing, for example, the score-based anomaly detection processing method in the above embodiment can be used to obtain anomaly detection results. The anomaly detection results are, for example, data anomalies at a certain time point in the time series data to be detected.

[0134] The industrial Internet anomaly detection method provided in the embodiment of the present application obtains the time series data to be detected, which is obtained by collecting the operating data of the target task of the industrial Internet; inputs the time series data to be detected into the domain projection module of the anomaly detection model to obtain the target features of the target dimension corresponding to the time series data to be detected output by the domain projection module; inputs the target features into the anomaly detection module of the anomaly detection model for anomaly detection processing to obtain the anomaly detection results corresponding to the time series data to be detected. Since the anomaly detection model in the embodiment of the present application is a model obtained through multi-task learning training and has good generalization ability, the anomaly detection model can more accurately detect anomalies in the time series data corresponding to different tasks in the industrial Internet, and can improve the performance of processing the task of anomaly detection of time series data, and has universality and practicality.

[0135] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0136] Figure 8This is a schematic diagram of the structure of a model training device provided in an embodiment of the present application, which is used to train an anomaly detection model. The anomaly detection model includes a domain projection module, a feature mixing enhancement module, an anomaly detection module, and a feature alignment module. Figure 8 As shown, the model training device 800 of the embodiment of the present application includes: an acquisition unit 801, which is used to acquire training samples, and the training samples include multiple time series data samples. The multiple time series data samples are obtained by collecting the operation data of different tasks when different tasks of the industrial Internet are in normal operation; a feature extraction unit 802, which is used to input the multiple time series data samples into the domain projection module to extract features of a unified dimension, and obtain the target features of the target dimension corresponding to each time series data sample output by the domain projection module; a feature mixing enhancement unit 803, which is used to input the multiple target features into the feature mixing enhancement module for feature mixing enhancement processing, and obtain the output of the feature mixing enhancement module Positive features and negative features of multiple target dimensions; an anomaly detection unit 804, used to input the target features, positive features and negative features into the anomaly detection module for anomaly detection processing, and obtain the inter-block representation and intra-block representation corresponding to the target features, positive features and negative features, respectively, and the representation loss between the inter-block representation and the intra-block representation; a determination unit 805, used to determine the target loss based on the first alignment loss between the multiple target features, the second alignment loss between the multiple intra-block representations and the representation loss, the first alignment loss and the second alignment loss are obtained through the feature alignment module; an adjustment unit 806, used to adjust the parameters of the anomaly detection model according to the target loss to obtain a trained anomaly detection model.

[0137] Optionally, the feature mixing enhancement unit 803 can be specifically used to: perform a summation process between two or more target features to obtain a positive feature; perform an insertion process of abnormal feature fragments within the features of the multiple target features to obtain a negative feature.

[0138] Optionally, the determination unit 805 can be specifically used to: input multiple target features into the feature alignment module for head alignment processing to obtain a first alignment loss; input multiple intra-block representations into the feature alignment module for tail alignment processing to obtain a second alignment loss; determine the target loss as the sum of the first alignment loss, the second alignment loss and the representation loss.

[0139] Optionally, the domain projection module includes a first projector and a second projector, and the feature extraction unit 802 can be specifically used to: input the time series data sample into the first projector for dimensional unification processing, and perform feature extraction to obtain the initial features of the target dimension output by the first projector; input the initial features into the second projector for feature extraction to obtain the target features output by the second projector.

[0140] Optionally, the anomaly detection unit 804 can be specifically used to: for each feature of the target feature, the positive feature and the negative feature, perform block processing on the feature to obtain block-processed features; based on the block-processed features, obtain inter-block input features and intra-block input features; perform attention weight normalization and upsampling on the inter-block input features and the intra-block input features to obtain inter-block representations and intra-block representations; and obtain representation loss based on the difference between the inter-block representations and the intra-block representations.

[0141] The device of the embodiment of the present application can be used to execute the model training method in any of the above method embodiments. Its implementation principle and technical effects are similar and will not be repeated here.

[0142] Fig. 9 A schematic diagram of the structure of an industrial Internet anomaly detection device provided in an embodiment of the present application is shown in FIG. Fig. 9 As shown, the industrial Internet anomaly detection device 900 of the embodiment of the present application includes: an acquisition unit 901, which is used to acquire time series data to be detected, and the time series data to be detected is obtained by collecting the operation data of the target task of the industrial Internet; a feature extraction unit 902, which is used to input the time series data to be detected into the domain projection module of the anomaly detection model to extract features of a unified dimension, and obtain the target features of the target dimension corresponding to the time series data to be detected output by the domain projection module, and the anomaly detection model is trained using the model training method in any of the above method embodiments; an anomaly detection unit 903, which is used to input the target features into the anomaly detection module of the anomaly detection model to perform anomaly detection processing, and obtain the anomaly detection results corresponding to the time series data to be detected.

[0143] The device of the embodiment of the present application can be used to execute the solution of the industrial Internet anomaly detection method in any of the above-mentioned method embodiments. Its implementation principle and technical effects are similar and will not be repeated here.

[0144] Fig.10 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Fig.10 As shown, the electronic device 1000 may include: at least one processor 1001 and a memory 1002 .

[0145] The memory 1002 is used to store programs. Specifically, the programs may include program codes, and the program codes include computer-executable instructions.

[0146] The memory 1002 may include a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0147] The processor 1001 is used to execute the computer execution instructions stored in the memory 1002 to implement the model training method or the industrial Internet anomaly detection method described in the aforementioned method embodiment. Among them, the processor 1001 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. Specifically, when implementing the model training method or the industrial Internet anomaly detection method described in the aforementioned method embodiment, the electronic device can be, for example, an electronic device with processing functions such as a server.

[0148] Optionally, the electronic device 1000 may further include a communication interface 1003. In a specific implementation, if the communication interface 1003, the memory 1002 and the processor 1001 are implemented independently, the communication interface 1003, the memory 1002 and the processor 1001 may be interconnected through a bus and communicate with each other. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc., but it does not mean that there is only one bus or one type of bus.

[0149] Optionally, in a specific implementation, if the communication interface 1003, the memory 1002 and the processor 1001 are integrated on a chip, the communication interface 1003, the memory 1002 and the processor 1001 can communicate through an internal interface.

[0150] The present application also provides a computer-readable storage medium, which stores computer program instructions. When a processor executes the computer program instructions, the above-mentioned model training method solution or industrial Internet anomaly detection method solution is implemented.

[0151] The present application also provides a computer program product, including a computer program, which, when executed, implements the above-mentioned model training method solution or industrial Internet anomaly detection method solution.

[0152] The computer-readable storage medium mentioned above may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. The readable storage medium may be any available medium that can be accessed by a general or special purpose computer.

[0153] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in a dedicated integrated circuit. Of course, the processor and the readable storage medium can also exist as discrete components in a model training device or an industrial Internet anomaly detection device.

[0154] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0155] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A model training method, characterized in that: Used to train an anomaly detection model, the anomaly detection model includes a domain projection module, a feature mixing enhancement module, an anomaly detection module and a feature alignment module, and the model training method includes: Acquire training samples, where the training samples include a plurality of time series data samples, where the plurality of time series data samples are obtained by respectively collecting operation data of different tasks of the industrial Internet when the different tasks are in a normal operating state; Input the multiple time series data samples into the domain projection module to extract features of a unified dimension, and obtain a target feature of a target dimension corresponding to each of the time series data samples output by the domain projection module; Inputting the plurality of target features into the feature mixing enhancement module for feature mixing enhancement processing, and obtaining the positive features and negative features of the plurality of target dimensions output by the feature mixing enhancement module; Inputting the target feature, the positive feature, and the negative feature into the anomaly detection module for anomaly detection processing, obtaining inter-block representations and intra-block representations corresponding to the target feature, the positive feature, and the negative feature, respectively, and a representation loss between the inter-block representation and the intra-block representation; Determine a target loss based on a first alignment loss between the plurality of target features, a second alignment loss between the plurality of intra-block representations, and the representation loss, wherein the first alignment loss and the second alignment loss are obtained by the feature alignment module; The parameters of the anomaly detection model are adjusted according to the target loss to obtain the trained anomaly detection model.

2. The model training method according to claim 1, characterized in that: The step of inputting the plurality of target features into the feature mixing enhancement module for feature mixing enhancement processing to obtain the plurality of positive features and negative features of the target dimensions output by the feature mixing enhancement module includes: Performing a summation process between two of the plurality of target features to obtain the positive feature; Insertion processing of abnormal feature fragments within the features is performed on the plurality of target features to obtain the negative features.

3. The model training method according to claim 1, characterized in that: The determining of the target loss based on the first alignment loss between the plurality of target features, the second alignment loss between the plurality of in-block representations, and the representation loss comprises: Inputting the plurality of target features into the feature alignment module for head alignment processing to obtain the first alignment loss; Inputting the plurality of intra-block representations into the feature alignment module for tail alignment processing to obtain the second alignment loss; The target loss is determined as a sum of the first alignment loss, the second alignment loss, and the characterization loss.

4. The model training method according to any one of claims 1 to 3, characterized in that: The domain projection module includes a first projector and a second projector, and the inputting of the plurality of time series data samples into the domain projection module to extract features of a unified dimension, and obtaining a target feature of a target dimension corresponding to each of the time series data samples output by the domain projection module, includes: Inputting the time series data sample into the first projector for dimension unification processing and feature extraction to obtain the initial features of the target dimension output by the first projector; The initial features are input into the second projector for feature extraction to obtain the target features output by the second projector.

5. The model training method according to any one of claims 1 to 3, characterized in that: The step of inputting the target feature, the positive feature, and the negative feature into the anomaly detection module for anomaly detection processing to obtain the inter-block representation and the intra-block representation corresponding to the target feature, the positive feature, and the negative feature, respectively, and the representation loss between the inter-block representation and the intra-block representation, includes: For each of the target features, the positive features and the negative features, the features are processed in blocks to obtain block-processed features; based on the block-processed features, inter-block input features and intra-block input features are obtained; attention weight normalization and upsampling are performed on the inter-block input features and the intra-block input features to obtain the inter-block representation and the intra-block representation; and the representation loss is obtained according to the difference between the inter-block representation and the intra-block representation.

6. An industrial Internet anomaly detection method, characterized in that: include: Acquire time series data to be detected, where the time series data to be detected is obtained by collecting operation data of target tasks of the industrial Internet; Inputting the time series data to be detected into a domain projection module of an anomaly detection model to extract features of a unified dimension, and obtaining target features of a target dimension corresponding to the time series data to be detected output by the domain projection module, wherein the anomaly detection model is trained using the model training method according to any one of claims 1 to 5; The target feature is input into the anomaly detection module of the anomaly detection model for anomaly detection processing to obtain an anomaly detection result corresponding to the time series data to be detected.

7. A model training device, characterized in that: Used to train an anomaly detection model, the anomaly detection model includes a domain projection module, a feature mixing enhancement module, an anomaly detection module and a feature alignment module, and the model training device includes: An acquisition unit, used to acquire training samples, wherein the training samples include a plurality of time series data samples, and the plurality of time series data samples are obtained by respectively collecting operation data of different tasks of the industrial Internet when the different tasks are in a normal operating state; A feature extraction unit, configured to input the plurality of time series data samples into the domain projection module to perform feature extraction of a unified dimension, and obtain a target feature of a target dimension corresponding to each of the time series data samples output by the domain projection module; A feature mixing and enhancing unit, used for inputting the plurality of target features into the feature mixing and enhancing module for performing feature mixing and enhancing processing, and obtaining the positive features and negative features of the plurality of target dimensions output by the feature mixing and enhancing module; an anomaly detection unit, used for inputting the target feature, the positive feature and the negative feature into the anomaly detection module for anomaly detection processing, and obtaining the inter-block representation and the intra-block representation corresponding to the target feature, the positive feature and the negative feature, respectively, and the representation loss between the inter-block representation and the intra-block representation; a determining unit, configured to determine a target loss based on a first alignment loss between the plurality of target features, a second alignment loss between the plurality of intra-block representations, and the representation loss, wherein the first alignment loss and the second alignment loss are obtained by the feature alignment module; An adjustment unit is used to adjust the parameters of the anomaly detection model according to the target loss to obtain the trained anomaly detection model.

8. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed, the method according to any one of claims 1 to 6 is implemented.

Images