Anomaly detection method for sensor data based on self-supervision and hybrid expert model

Abstract

The present invention discloses a sensor data anomaly detection method based on self-supervision and a hybrid expert model. This method belongs to the field of device sensor data anomaly detection. Using a hybrid expert model structure, data is assigned to different expert submodules for learning during training. Each expert submodule processes only a portion of normal data, resulting in more accurate learning of the decision boundaries of normal data, particularly when normal data contains a variety of different data distributions. Compared to using a large model to learn all data, the hybrid expert model can effectively improve the model's ability to identify abnormal data. By designing and constructing a variety of self-supervised transformation data and self-supervised auxiliary tasks, the model is forced to behave differently from normal data when the self-supervised transformation data is transformed, minimizing the problem of the model displaying similar characteristics to normal data when the abnormal data exhibits characteristics similar to those of the normal data.

Description

Technical Field

[0001] The present invention belongs to the field of device sensor data anomaly detection, and more specifically, relates to a sensor data anomaly detection method based on self-supervision and hybrid expert model. Background Art

[0002] Anomaly detection, that is, detecting data that does not behave as expected in a large amount of data, can help detect early signs of equipment degradation and failure, and then implement corresponding inspection and maintenance strategies to avoid greater harm.

[0003] Because obtaining complete anomaly labels is difficult in the real world, anomaly detection methods primarily rely on learning from large amounts of unlabeled historical data. These methods are known as unsupervised anomaly detection methods. Because anomaly data is rare, many unsupervised anomaly detection methods ignore the small amount of anomalies present in unlabeled data and treat them as normal data for training.

[0004] Due to the powerful expressive power of deep learning, deep anomaly detection methods often demonstrate superior performance compared to traditional methods. Widely used deep unsupervised anomaly detection methods include reconstruction-based autoencoder methods and deep support vector data description (DSD)-based methods. Autoencoder methods assume that the reconstruction error of normal data is small and the reconstruction error of abnormal data is large. During testing, the size of the data reconstruction error is used to determine whether it is an anomaly. Deep support vector data description methods map data to a feature space and train the feature space to minimize the volume of a hypersphere containing normal data. It assumes that normal data is inside the hypersphere and abnormal data is outside the hypersphere in the feature space. During testing, the distance from the data feature space to the center of the hypersphere is used to determine whether the data is abnormal. Current studies have proposed deep autoencoding support vector data description (DSVDD) that combines autoencoders and deep support vector data description for anomaly detection.

[0005] However, these models are all trained on normal data, learning its characteristics and assuming that abnormal data exhibits different characteristics from normal data, thus enabling anomaly detection. However, this assumption is not always true. Because the models themselves have a certain degree of generalization ability and some abnormal data is not very different from normal data, the model's performance on some abnormal data may be similar to that on normal data. This, in turn, results in insufficient differentiation of some abnormal data and poor anomaly detection performance. Summary of the Invention

[0006] In response to the above-mentioned defects or improvement needs of the existing technology, the present invention provides a sensor data anomaly detection method based on self-supervision and hybrid expert models, thereby solving the technical problem that the existing anomaly detection model exhibits similar characteristics to normal data for some abnormal data, resulting in insufficient ability of the model to distinguish some abnormal data and poor anomaly detection performance.

[0007] To achieve the above objectives, according to a first aspect of the present invention, a sensor data anomaly detection method based on self-supervision and hybrid expert model is provided, comprising:

[0008] Training phase:

[0009] S1, adding a self-supervision module after any potential layer or output layer of a deep unsupervised model including M consecutive network structure units, and replacing the N consecutive network structure units of the deep unsupervised model with a hybrid expert module to obtain an anomaly detection model; the hybrid expert module includes a gating network G and E expert submodules controlled by G, each expert submodule includes the N consecutive network structure units, and the gating network G is used to determine the activated expert submodule based on the routing weights of each expert submodule, where N<M;

[0010] S2, taking the sum of the loss function of the anomaly detection model, the self-supervision loss function, and the gating network loss function as the total loss function, and training the anomaly detection model using the target training set with the goal of minimizing the total loss function;

[0011] S3, testing the trained anomaly detection model based on the test set to determine the anomaly score threshold;

[0012] Application stage:

[0013] The sensor data to be detected is input into the trained anomaly detection model, and the anomaly score of the sensor data to be detected is calculated based on the output of the anomaly detection model. If the anomaly score is higher than the anomaly score threshold, the detection result is abnormal, otherwise it is normal.

[0014] According to a second aspect of the present invention, there is provided a sensor data anomaly detection system based on self-supervision and hybrid expert models, comprising: a computer-readable storage medium and a processor;

[0015] The computer-readable storage medium is used to store executable instructions;

[0016] The processor is configured to read the executable instructions stored in the computer-readable storage medium and execute the method according to the first aspect.

[0017] According to a third aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to execute the method according to the first aspect.

[0018] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0019] The method proposed in this paper uses a hybrid expert model architecture, distributing data to different expert submodules during training. Each expert submodule processes only a portion of normal data, leading to more accurate learning of the decision boundary for normal data, especially when normal data contains a variety of different data distributions. Compared to using a large model to learn all the data, the hybrid expert model can effectively improve the model's ability to identify abnormal data.

[0020] The method provided by the present invention, by designing and constructing a variety of self-supervised transformation data and self-supervised auxiliary tasks, forces the model to behave differently from normal data as much as possible for self-supervised transformation data, and avoids the problem that the model behaves similarly to normal data for abnormal data. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 One of the structural diagrams of the sensor data anomaly detection model based on self-supervision and hybrid expert model provided in an embodiment of the present invention;

[0022] Figure 2 The second structural diagram of the sensor data anomaly detection model based on self-supervision and hybrid expert model provided by an embodiment of the present invention;

[0023] Figure 3 A training flowchart of a sensor data anomaly detection model based on self-supervision and a hybrid expert model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0025] An embodiment of the present invention provides a sensor data anomaly detection method based on self-supervision and hybrid expert model, such as comprising:

[0026] Training phase:

[0027] S1, build anomaly detection model: Figure 1As shown, a self-supervision module is added after any potential layer or output layer of a deep unsupervised model including M consecutive network structure units, and a hybrid expert module is used to replace the N consecutive network structure units of the deep unsupervised model to obtain an anomaly detection model; the hybrid expert module includes a gating network G and E expert submodules controlled by G, each expert submodule includes the N consecutive network structure units, and the gating network G is used to determine the activated expert submodule according to the routing weight of each expert submodule, N<M;

[0028] S2, training anomaly detection model: taking the sum of the loss function of the anomaly detection model, the self-supervision loss function, and the gating network loss function as the total loss function, and minimizing the total loss function as the goal, and training the anomaly detection model using the target training set;

[0029] S3, test the anomaly detection model: test the trained anomaly detection model based on the test set to determine the anomaly score threshold.

[0030] Furthermore, the target training set includes an initial training set and simulated abnormal data obtained by transforming the initial training set, and the initial training set includes normalized historical normal data of the sensor;

[0031] The test set includes normalized historical normal data and historical abnormal data of the sensor;

[0032] The dimension of each data in the initial training set, target training set and test set is the number of sensors;

[0033] In the application stage, the sensor data to be detected is normalized and then input into the trained anomaly detection model to obtain the anomaly detection results.

[0034] Furthermore, the simulated abnormal data obtained by transforming the initial training set is specifically:

[0035] For any data in the initial training set: randomly mask (i.e., set to zero) a preset proportion of its dimensional attribute values, and / or randomly multiply a preset proportion of its dimensional attribute values ​​by a first preset multiple, and / or randomly multiply a preset proportion of its dimensional attribute values ​​by a second preset multiple, to obtain simulated abnormal data;

[0036] The first preset multiple is a positive number, and the second preset multiple is a negative number.

[0037] Furthermore, the deep unsupervised model is a deep autoencoding support vector data description model, a deep autoencoding model or a deep support vector data description model.

[0038] Furthermore, the loss of the deep autoencoder support vector data description model includes the reconstruction error loss of the training data and the minimization loss of the distance from the feature space of the initial training data set to the center of the hypersphere, the loss of the deep autoencoder model only includes the reconstruction error loss, and the loss of the deep support vector data description model only includes the minimization loss of the distance from the feature space of the initial training data set to the center of the hypersphere; the center of the hypersphere is the mean of the feature vectors of the feature space of the initial training set;

[0039] When the deep unsupervised model is a deep support vector data description model, the self-supervised loss is the cross entropy loss for classifying simulated abnormal data; when the deep unsupervised model is a deep autoencoder support vector data description model or a deep autoencoder model, the self-supervised loss also includes a mask recovery loss;

[0040] Regardless of whether the deep unsupervised model is a deep autoencoding support vector data description model, a deep autoencoding model, or a deep support vector data description model, the gating network loss includes importance loss and load loss; wherein the importance loss is the variance of the routing weights of the gating network for all expert sub-modules, and the load loss is the variance of the selection probability of all expert sub-modules.

[0041] Specifically, before building the anomaly detection model (ie, step S1), a training set and a test set are first built, and data preprocessing is performed.

[0042] Collect historical data from multiple sensors and divide them into training and testing datasets. The training datasets are all normal samples, and the testing datasets are labeled and contain abnormal samples. The dimension of each data in the dataset is the number of sensors; for example, if the device includes 6 sensors, each data is (X1, X2, X3, X4, X5, X6).

[0043] That is, historical data of the sensor under normal working conditions and historical data of the sensor under abnormal conditions are collected, and 80% of the data under normal working conditions are randomly selected as the training data set, and the other 20% and the data under abnormal conditions together constitute the test data set.

[0044] Dataset preprocessing: Perform Min-Max normalization on the training dataset and record the maximum value Max_Val of the i-th sensor in the training dataset i and the minimum value Min_Val i , for each data in the training dataset and the test dataset, the i-th sensor data x i , using the formula Update, when Max_Val i =Min_Val i When x i =0.5.

[0045] By performing T self-supervised transformations on the training dataset, we construct T self-supervised transformed datasets to simulate potential unknown anomalies. The self-supervised transformed data (i.e., simulated anomaly data) should be significantly different from the original training data (i.e., the initial training set). At the same time, we also normalize the data in the test set to highlight anomalies in the test set, making it easier for the model to identify them.

[0046] There are three types of self-supervised transformations: T = 3, which randomly masks a preset proportion of dimension attribute values ​​for each data point, setting the masked data to 0; randomly multiplies a preset proportion of feature attribute values ​​for each data point by a first preset multiple; and randomly multiplies a preset proportion of feature attribute values ​​for each data point by a second preset multiple. Because the training data has already been normalized, these three types of self-supervised transformations ensure that the distribution of the data differs from the training data in most cases. Sudden failure, sudden increase, or sudden decrease of sensor values ​​are common anomalies.

[0047] The value range of the preset ratio is 10%-30%, the value range of the first preset multiple is (1,10], and the value range of the second preset multiple is [-10,0].

[0048] For example, when the preset ratio is 25%, for sensor data with 4 dimensions (X1, X2, X3, X4), the data of any one dimension is set to 0, and / or the data of any one dimension is multiplied by a first preset multiple, and / or the data of any one dimension is multiplied by a second preset multiple.

[0049] S1. Build an anomaly detection model: The anomaly detection model should be built based on a deep unsupervised anomaly detection model (including M deep learning network structural units). The model should be composed of multiple deep learning network structural units, such as fully connected layers and attention modules. The model assumes that normal data has certain characteristic performance, such as the model's reconstruction error for normal data is small, and the model's feature space mapping of normal data is inside a hypersphere and close to the center of the hypersphere. It is assumed that abnormal data does not have this characteristic performance.

[0050] The deep unsupervised model can be a deep autoencoding support vector data description model, or a deep autoencoding model, or a deep support vector data description model.

[0051] For example, Figure 2As shown in the figure, the deep autoencoder support vector data description model selected is the deep autoencoder support vector data description model. The main structure of the model is an autoencoder structure, consisting of an encoder and a decoder. The encoder compresses the input data into a low-dimensional space and then passes it to the decoder, which outputs the reconstruction result. Both the encoder and decoder are composed of multiple layers of fully connected layers. The deep autoencoder support vector data description model assumes that the reconstruction error of normal data is smaller than that of abnormal data. At the same time, the compressed low-dimensional feature space of normal data is inside a hypersphere, while the abnormal data is outside the hypersphere.

[0052] It can be understood that if the deep unsupervised model is a deep autoencoding model, it only needs to satisfy the assumption that the reconstruction error of normal data is smaller than that of abnormal data; if the deep unsupervised model is a deep support vector data description model, it only needs to satisfy the assumption that the low-dimensional feature space of normal data is inside the hypersphere and the abnormal data is outside the hypersphere.

[0053] Adding self-supervision and hybrid expert modules: Based on the selected anomaly detection model, add self-supervision modules and hybrid expert modules.

[0054] The self-supervision module can be connected to a potential layer or output layer of the selected anomaly detection model in the form of a neural network, or it can be added to the loss function of the selected anomaly detection model in the form of a loss function term.

[0055] Furthermore, the self-supervised module includes a self-supervised auxiliary classification network structure, which is connected to the output of the encoder in the autoencoder structure. It is a classification network with a softmax layer as the last layer, which outputs the self-supervised label classification probability of the data; for the deep autoencoder support vector data description model and the deep autoencoder model, it also includes a self-supervised mask reconstruction loss term, which is calculated by the reconstruction error between the masked self-supervised transformed data and the original data before the transformation.

[0056] The hybrid expert module is used to replace the N consecutive network structure units in the selected model. These N network structure units are only part of the network structure of the model. The N network structure units of the selected model are copied E times as E expert sub-modules, and a trainable gated network G is added. The last layer of G is a softmax layer with a dimension of E. The input of G is the input of the original N network structure units, and the output of G is the routing weight of the expert sub-module. The k expert sub-modules with the largest routing weight are activated, and the input of G is connected to the input of the activated expert sub-module. The output of the activated expert sub-module is weighted and summed according to the routing weight as the output of the hybrid expert model.

[0057] S2, training anomaly detection model: constructing a model loss function based on the anomaly detection model constructed in step S1, and training the anomaly detection model based on the loss function, and saving the trained anomaly detection model.

[0058] The loss function includes the loss function of the anomaly detection model, the self-supervised loss function, and the gating network loss function, which respectively enable the model to learn the characteristics of normal data, enable the model to learn the performance of self-supervised transformed data that is different from normal training data, and enable the gating network G to uniformly activate each expert submodule for the training data set.

[0059] Furthermore, the goal of minimizing the loss function of the anomaly detection model constructed in step S1 is to minimize the training data reconstruction error and / or minimize the distance from the feature space to the center of the hypersphere.

[0060] The goal of minimizing the self-supervised loss function is to minimize the cross-entropy loss for classifying different self-supervised transformed data. For the deep autoencoder support vector data description model and the deep autoencoder model, it also includes minimizing the reconstruction error of the masked self-supervised transformed data and the original data before the transformation. Among them, minimizing the cross-entropy loss for classifying different self-supervised transformed data is to make the latent layer features of the self-supervised transformed data clearly distinguishable from the normal training data through back propagation, and to enable the model to learn the true latent semantic features of the normal data, rather than just learning the copy operation of the input data. Minimizing the reconstruction error of the masked self-supervised transformed data and the original data before the transformation is a supervised way to avoid the model's reconstruction error of abnormal data being very small.

[0061] The gating network loss function includes importance loss and load loss. The goal of minimizing importance loss is to minimize the variance of the routing weights of the gating network for all experts in a batch of data. The goal of minimizing load loss is to minimize the variance of the selection probabilities of all expert submodules in a batch of data. Load loss is added because even in some cases where the overall routing weights are balanced, some expert submodules may still obtain all the allocations.

[0062] S3, testing the anomaly detection model: Test the trained anomaly detection model based on the test set, calculate the anomaly score based on the model's performance on the test set data, set the anomaly score threshold and iterate continuously, calculate the F1 Score performance evaluation indicator of welding anomaly detection under different anomaly thresholds, and finally save the anomaly score threshold corresponding to the highest F1 Score.

[0063] When the deep unsupervised model is a deep autoencoder support vector data description model, the data anomaly score is obtained by the weighted sum of the data reconstruction error and the distance from the feature space to the center of the hypersphere; when the deep unsupervised model is a deep autoencoder model, the data anomaly score is the data reconstruction error; when the deep unsupervised model is a deep support vector data description model, the data anomaly score is the distance from the feature space to the center of the hypersphere. The threshold is initially set to the highest anomaly score value of the training data. Then, the training data and test data are treated as a whole. The anomaly score of the data with an anomaly score higher than the current threshold is first continuously used as the new threshold. The F1Score result of each iteration is calculated, and the optimal result and threshold are recorded until the data with the highest anomaly score is iterated. Then, starting from the highest anomaly score value of the training data, the anomaly score of the data with anomaly score lower than the current threshold is continuously used as the new threshold. The F1 Score result of each iteration is calculated, and the optimal result and threshold are recorded until the data with the lowest anomaly score is iterated.

[0064] Application stage:

[0065] The sensor data to be detected is input into the trained anomaly detection model to obtain the feature representation of the sensor data to be detected, and the anomaly score of the sensor data to be detected is calculated based on the feature representation. If the anomaly score is higher than the anomaly score threshold, the detection result is abnormal, otherwise it is normal.

[0066] Specifically, when performing sensor data anomaly detection, new sensor data q is obtained, and for the i-th sensor data q of the data i ,implement When Max_Val i =Min_Val i When q i =q i -Max_Val i +0.5, input the processed model into the trained model, calculate the anomaly score of the data based on the model's characteristic performance of the sensor data, and compare it with the saved anomaly score threshold. If it is higher, it is judged as abnormal, and if it is lower, it is normal.

[0067] It can be understood that for the deep autoencoder support vector data description model, the feature representation of the data is the latent layer feature representation and the reconstructed output of the autoencoder; for the deep autoencoder model, the feature representation of the data is the reconstructed output; for the deep support vector data description model, the output is the latent layer feature representation, that is, the feature representation of the data is the latent layer feature representation.

[0068] Among them, the output potential layer feature representation is used to calculate the distance from the feature space to the center of the hypersphere; the reconstruction output is used to calculate the reconstruction error value.

[0069] The method provided by the present invention is further described below with a specific example, which comprises the following steps:

[0070] 1) Collect historical data from multiple sensors under normal operating conditions and when they experience abnormalities. Randomly select 80% of the normal operating data as the training dataset, and the remaining 20% ​​along with the abnormal operating data to form the test dataset. The training dataset consists of normal samples by default, while the test dataset is labeled. The dimension of each data point in the dataset is the number of sensors.

[0071] 2) Dataset preprocessing: perform Min-Max normalization on the training dataset and record the maximum value Min_Val of the i-th sensor in the training dataset. i and the minimum value Min_Val i , for each data x in the training dataset and the test dataset, the i-th sensor output x i , using the formula Update. T self-supervised datasets are constructed by performing T = 3 self-supervised transformation operations on the normalized training dataset. The self-supervised data are used to simulate potential unknown anomalies. The T self-supervised transformation operations are as follows: 25% of the dimensional attribute values ​​of each data are randomly masked, that is, the masked data are set to 0, 25% of the feature attribute values ​​of each data are randomly multiplied by 5.0, and 25% of the feature attribute values ​​of each data are randomly multiplied by -1.0.

[0072] 3) Anomaly detection model selection: The deep autoencoder support vector data description model was selected. The model's main structure is an autoencoder, consisting of an encoder and a decoder. The encoder compresses the input data into a low-dimensional space and then passes it to the decoder, which outputs the reconstruction result. Both the encoder and decoder are composed of multiple fully connected layers. The deep autoencoder support vector data description model assumes that the reconstruction error of normal data is smaller than that of abnormal data. At the same time, the compressed low-dimensional feature space of normal data lies within a hypersphere, while the abnormal data lies outside the hypersphere.

[0073] 4) Adding self-supervision and hybrid expert modules: Based on the selected anomaly detection model, add self-supervision modules and hybrid expert modules to obtain the sensor data anomaly detection model based on self-supervision and hybrid expert modules, as shown in Figure 1As shown. In this embodiment, a self-supervised auxiliary classification network structure is added. The self-supervised auxiliary classification structure is connected to the output of the encoder in the autoencoder structure. It is a classification network. The last layer is a softmax layer, which outputs the self-supervised label classification probability of the data. A self-supervised mask reconstruction loss term is also added to the loss function. The loss term is calculated by the reconstruction error between the masked self-supervised transformed data and the original data before the transformation. The hybrid expert module is used to replace the N consecutive fully connected layers in the model. These N fully connected layers are part of the network structure of the model. The N fully connected layers of the selected model are copied E times as E expert sub-modules. A trainable gated network G is added. The last layer of G is a softmax layer with a dimension of E. The input of G is the input of the original N fully connected layers. The output of G is the routing weight of the expert sub-module. The k expert sub-modules with the largest routing weights are activated. The input of G is connected to the input of the activated expert sub-module. The output of the activated expert sub-module is weighted and summed according to the routing weight as the output of the hybrid expert model. The schematic diagram of the model structure of this embodiment is shown in Figure 2 shown.

[0074] 5) Construct a model loss function based on the anomaly detection model described in step S4, and train the anomaly detection model based on the loss function. The known conditions of the training phase are as follows: There are T+1 datasets of self-supervised transformed data (i.e., simulated anomaly data obtained by transforming the initial training set) and normalized original data (i.e., the initial training set). During training, a batch of data consists of the original data and T self-supervised transformed data of the original data.

[0075] The loss function of the deep autoencoder support vector data description model includes the reconstruction error minimization loss L MSE_NOR , the distance from the feature space to the center of the hypersphere minimizes the loss L DSVDD , L MSE_NOR Calculated from the reconstruction error of the original training data, L DSVDD It is calculated from the distance from the original training data feature space to the center of the hypersphere, and the center of the hypersphere is set to the mean of all normalized feature values ​​of the original training data feature space.

[0076] It is understandable that when the deep unsupervised model is a deep autoencoding model, its loss function does not have L DSVDD Loss; When the deep unsupervised model is a deep support vector data description model, its loss function does not have L MSE_NOR and L MSE_S loss.

[0077] The self-supervised loss function includes the cross entropy loss L for classifying different self-supervised transformed data CE and mask recovery loss L MSE_S , LMSE_S It is calculated by the reconstruction error between the reconstructed output of the masked self-supervised transformation type data and the original normal data.

[0078] For example: Assume that the original data is X = (X1, X2, X3, X4, X5, X6), and the masked data is Xy = (Xy1, Xy2, Xy3, Xy4, Xy5, Xy6). Input Xy into Figure 2 The deep autoencoding support vector data description model shown (including encoder, decoder, hybrid expert module and self-supervision module) obtains the reconstructed output as Xy'=(Xy1',Xy2',Xy3',Xy4',Xy5',Xy6'); then L MSE_S is the reconstruction error of X and Xy'.

[0079] The gating network loss function includes the importance loss (Importance Loss) L Imp and load loss (LoadLoss) L load , the importance loss encourages the balanced use of experts, for the expert submodule e i First, calculate the expert submodule e corresponding to all data in each batch i Assuming that the number of expert submodules is E, the normalized routing weight values ​​of all expert submodules form a set, and the importance loss of the batch data is L Imp is the variance of the data set. The importance loss is intended to ensure that all expert submodules have similar output routing weights on average. However, in some cases, the average routing weight does not completely determine the final expert submodule allocation. In some cases where the overall routing weight is balanced, some expert submodules may still receive all allocations. Therefore, there is a load loss, which is used to calculate the expert submodule e i The probability of being selected and the average probability of being selected of all expert submodules are forced to be similar. i The probability of being selected can be approximated as a batch of submodules e i The number of times it is activated, that is, frequency is used instead of probability. For each input x, a score threshold is defined. If the routing weight value of the expert submodule is greater than or equal to the threshold, the expert submodule is activated. This is simply defined as the kth largest gate routing weight score, that is, k expert submodules are activated each time. So for expert e i The activation probability value in a batch is the number of times the expert is activated. Finally, the activation probability values ​​of all expert submodules are constructed into a set, and the load loss term is L load is the variance of the set data, and the gate loss function is the mean of the importance loss and the load loss Finally, the loss function formula for each batch of data is Ltotal =L MSE_NOR +λL DSVDD +L MSE_S +L CE +γ g L g ,λ,γ g It is a weight hyperparameter, which is set manually before training. λ defaults to 1, and γ g The default value is 0.01. The training strategy flow chart of the embodiment is as follows Figure 3 As shown, Figure 3 An optional pre-training step is included. The pre-training step is mainly to better determine the center of the hypersphere. The pre-training loss function is less L than the overall loss function. DSVDD Term, pre-training loss function formula L pretrain =L MSE_NOR +L MSE_S +L CE +γ g L g After the model training is completed, save the trained model, including the model's network parameters and the center value of the hypersphere used in training.

[0080] It can be understood that for each piece of data in a batch, some expert sub-modules will be activated to perform specialized learning on them.

[0081] 6) The abnormal score of the data is obtained by the weighted sum of the reconstruction error of the data and the distance from the feature space to the center of the hypersphere. Assume that the reconstruction error of the data is S MSE_NOR , the distance from the data feature space to the center of the hypersphere is S DSVDD , the final abnormal score of the data is S = S MSE_NOR +λS DSVDD λ is a weight hyperparameter, consistent with the setting in step S5. The anomaly score threshold is initially set to the highest anomaly score value of the training data. The training and test data are then treated as a whole. The anomaly scores of data with anomaly scores higher than the current threshold are first used as the new threshold. The F1 Score results for each iteration are calculated, and the optimal result and threshold are recorded until the data with the highest anomaly score is found. Then, starting from the highest anomaly score value of the training data, the anomaly scores of data with anomaly scores lower than the current threshold are used as the new threshold. The F1 Score results for each iteration are calculated, and the optimal result and threshold are recorded until the data with the lowest anomaly score is found. Finally, the anomaly score threshold corresponding to the highest F1 Score is saved.

[0082] 7) When performing sensor data anomaly detection, obtain new sensor data q, and for the i-th sensor data q of the data i ,implement When Max_Vali =Min-Val i When q i =q i -Max_Val i +0.5, input the processed data into the trained model, and calculate the weighted sum of the reconstruction error value of the data and the distance from the data feature space to the center of the hypersphere as the anomaly score. The specific calculation is the same as step S6. The anomaly score of the data is compared with the saved anomaly score threshold. If it is higher, it is considered abnormal; if it is lower, it is normal.

[0083] In summary, the method provided by this invention simulates unknown anomalies by constructing a variety of self-supervised transformed data, then using supervised learning to force the model to represent the self-supervised transformed data differently from the original data. Each expert submodule in the hybrid expert model learns only from a subset of normal data, enabling the model to more accurately learn the features of normal data. This enhances the model's ability to distinguish between normal and abnormal data, especially when the normal data contains multiple different distributions.

[0084] An embodiment of the present invention provides a sensor data anomaly detection system based on self-supervision and hybrid expert model, comprising: a computer-readable storage medium and a processor;

[0085] The computer-readable storage medium is used to store executable instructions;

[0086] The processor is configured to read the executable instructions stored in the computer-readable storage medium and execute the method described in any one of the above embodiments.

[0087] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to execute the method described in any one of the embodiments.

[0088] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A sensor data anomaly detection method based on self-supervision and hybrid expert model, characterized in that: include: Training phase: S1, adding a self-supervision module after any potential layer or output layer of a deep unsupervised model including M consecutive network structure units, and replacing the N consecutive network structure units of the deep unsupervised model with a hybrid expert module to obtain an anomaly detection model; the hybrid expert module includes a gating network G and E expert submodules controlled by G, each expert submodule includes the N consecutive network structure units, and the gating network G is used to determine the activated expert submodule based on the routing weights of each expert submodule, where N<M; S2, taking the sum of the loss function of the anomaly detection model, the self-supervision loss function, and the gating network loss function as the total loss function, and training the anomaly detection model using the target training set with the goal of minimizing the total loss function; S3, testing the trained anomaly detection model based on the test set to determine the anomaly score threshold; Application stage: The sensor data to be detected is input into the trained anomaly detection model, and the anomaly score of the sensor data to be detected is calculated based on the output of the anomaly detection model. If the anomaly score is higher than the anomaly score threshold, the detection result is abnormal, otherwise it is normal.

2. The method according to claim 1, wherein The target training set includes an initial training set and simulated abnormal data obtained by transforming the initial training set, and the initial training set includes normalized historical normal data of the sensor; The test set includes normalized historical normal data and historical abnormal data of the sensor; The dimension of each data in the initial training set, target training set and test set is the number of sensors; In the application stage, the sensor data to be detected is normalized and then input into the trained anomaly detection model to obtain the anomaly detection results.

3. The method according to claim 2, wherein The simulated abnormal data obtained by transforming the initial training set is specifically: For any data in the initial training set: randomly masking a preset proportion of its dimensional attribute values, and / or randomly multiplying a preset proportion of its dimensional attribute values ​​by a first preset multiple, and / or randomly multiplying a preset proportion of its dimensional attribute values ​​by a second preset multiple, to obtain simulated abnormal data; The first preset multiple is a positive number, and the second preset multiple is a negative number.

4. The method according to claim 1, wherein The deep unsupervised model is a deep autoencoding support vector data description model, a deep autoencoding model or a deep support vector data description model.

5. The method according to claim 4, wherein When the deep unsupervised model is a deep autoencoder support vector data description model, the anomaly score of the data is obtained by the weighted sum of the reconstruction error of the data and the distance from the feature space to the center of the hypersphere; When the deep unsupervised model is a deep autoencoding model, the anomaly score of the data is the reconstruction error of the data; When the deep unsupervised model is a deep support vector data description model, the abnormal score of the data is the distance from the feature space to the center of the hypersphere.

6. The method according to claim 4 or 5, characterized in that When the deep unsupervised model is a deep autoencoding support vector data description model, its loss includes the reconstruction error loss of the training data and the loss of minimizing the distance from the feature space of the initial training data set to the center of the hypersphere; the self-supervised loss includes the cross entropy loss for classifying simulated abnormal data and the mask recovery loss; When the deep unsupervised model is a deep autoencoder model, its loss includes reconstruction error loss of training data; self-supervised loss includes cross entropy loss and mask recovery loss for classification of simulated abnormal data; When the deep unsupervised model is a deep support vector data description model, its loss includes minimizing the distance from the feature space of the initial training data set to the center of the hypersphere; the self-supervised loss includes the cross entropy loss for classifying simulated abnormal data; The center of the hypersphere is the mean of the feature vectors in the feature space of the initial training set; The gating network loss includes importance loss and load loss; wherein, the importance loss is the variance of the routing weights of the gating network for all expert submodules, and the load loss is the variance of the selection probabilities of all expert submodules.

7. A sensor data anomaly detection system based on self-supervision and hybrid expert model, characterized in that include: Computer-readable storage media and processor; The computer-readable storage medium is used to store executable instructions; The processor is configured to read the executable instructions stored in the computer-readable storage medium and execute the method according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to execute the method according to any one of claims 1 to 6.

Images