Industrial anomaly detection method based on time and channel double attention mechanism
By combining the industrial anomaly detection method of timing feature attention and channel attention mechanism, using the TEAM_DenseNet network model, the problem of difficulty in capturing complex time series characteristics in the prior art is solved, and higher anomaly detection accuracy and robustness are achieved.
Patent Information
- Application Number
- CN202411678566.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-22
- Publication Date
- 2025-03-28
AI Technical Summary
In existing industrial Internet of Things systems, anomaly detection methods are difficult to effectively capture the complex nonlinear and dynamic relationships in time series data, resulting in poor detection performance.
An industrial anomaly detection method combining timing feature attention and channel attention mechanism is adopted, and the time attention layer and channel attention layer are used to capture key timing feature and channel feature information in time series data through the TEAM_DenseNet network model.
This method can more effectively capture the relevant features in the time and channel dimensions, improve the accuracy and robustness of anomaly detection, and is suitable for various CNN architectures, with flexibility and high generalization capabilities.
Smart Images
Figure CN119720006A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of intelligent manufacturing and pattern recognition, and specifically relates to an industrial anomaly detection method based on a dual attention mechanism of time and channel. Background Art
[0002] Industrial Internet of Things (IIoT) refers to the use of sensors, devices and other Internet of Things (IoT) devices to provide network interconnection for intelligent industrial systems to achieve data collection, data analysis and automated control in industrial environments. It is the basis for the digital transformation journey of Industry 4.0 in traditional manufacturing, energy, power and other industries in recent years. However, although the IIoT can enhance the manufacturing process, abnormal behavior of devices as IIoT nodes may cause erroneous operations, affect the industrial production process, and cause significant economic losses. For example, due to the introduction of the IIoT, traditional isolated industrial systems are often exposed to public access, and devices that were originally designed with almost no security mechanisms are easily damaged by malicious users (such as attackers), causing abnormal device behavior. In addition, it is reported that nearly 3 billion liters of water are lost every day due to leakage in England and Wales, which also causes considerable economic losses to water companies. Therefore, it is of vital importance to use edge devices to collect sensor data from IIoT nodes, capture and intelligently monitor node behavior, and perform efficient and automated anomaly detection.
[0003] In industrial IoT systems, anomalies may be caused by factors such as sensor failures with mechanical problems, software anomalies, and human influence. Anomaly detection aims to discover specific patterns in the corresponding time series characteristics that do not conform to general rules and identify abnormal states in specific time steps. Generally speaking, anomaly detection algorithms can be divided into four types: rule-based processing methods, statistics-based methods, machine learning-based methods, and deep learning-based methods. Based on the rule processing method, this method requires obtaining rules in advance, and then diagnosing anomalies by determining whether the behavior is similar to the abnormal rules. However, this method is limited by expert knowledge and has problems such as imperfect rule bases. It cannot cope with new anomalies and has problems such as time-consuming rule matching. Statistical methods are suitable for low-dimensional data and data distribution can be correctly identified. There are two main methods: simple methods such as the 3σ criterion and Grubbs test, and more complex methods such as time series modeling (vector autoregression (VAR), autoregressive integrated moving average (ARIMA)). However, since this type of method usually assumes that the data obeys a certain time series distribution or a certain function, it makes them unable to capture the potential nonlinearity and dynamic relationships of complex data. Based on machine learning methods, there are three cases: supervised learning, unsupervised learning, and weakly supervised learning. However, in most practical cases, industrial IoT time series usually lack labels, and due to its inherent periodicity, trend, data imbalance and irregularity, labeling anomalies requires high expert costs and cannot guarantee the accuracy of anomaly labeling, so unsupervised learning is widely used. For example, based on statistical distribution (HBOS), based on distance (KNN), based on density estimation (LOF, LFCOF), based on clustering (EM, OC-SVM, SVDD. Although there are many unsupervised methods available, machine learning-based methods cannot handle the intrinsic characteristics of time series data, and these classic methods do not consider the correlation between time dependency and high-dimensional features, so there are problems such as poor anomaly detection performance.
[0004] In recent years, unsupervised anomaly detection methods based on deep learning have flourished in the field of anomaly detection in multivariate time series data of the Industrial Internet of Things because they can infer the correlation between time series and only require normal data without abnormal labeled data during training to achieve the purpose of identifying abnormal behavior.
[0005] Time series data is important information with time order collected within a specific time interval, in which the order and time correlation of data points are crucial. Time series classification (TSC) aims to classify data based on its characteristics that change over time. When dealing with a single data dimension, it is called univariate time series classification (UTSC); when dealing with multi-dimensional data, it is called multivariate time series classification (MTSC). TSC has a wide range of application value in many fields such as medical diagnosis and industrial monitoring, and plays an important role.
[0006] In the field of time series, traditional methods such as nearest neighbor (1-NN), dynamic time warping (DTW), time series forest, etc. have shown remarkable results on multiple benchmark datasets. However, these methods often have difficulty in capturing subtle temporal relationships in the time series of each variable, mainly due to the complexity of selecting appropriate features from a large set of features. In contrast, deep learning-based TSC algorithms show obvious advantages in this regard by effectively learning low-dimensional features without the burden of dealing with a large number of candidate features. Although deep learning models such as convolutional neural networks (CNNs) have been widely used in the field of TSC, they still have some limitations. Due to inherent architectural constraints, CNNs often have difficulty capturing long-term temporal dependencies. This limitation may lead to poor performance on time series data where temporal relationships are crucial. In addition, standard CNNs may not be able to effectively extract discriminative features from multivariate time series data, especially when dealing with complex patterns and interactions between different variables.
[0007] Recently, a model combining self-attention mechanism with CNN for time series classification has been proposed. The model usually consists of the following steps: (1) CNN feature extraction: extract local temporal features using one-dimensional convolutional layers; (2) self-attention mechanism: input the features extracted by CNN into the self-attention layer, calculate the attention weights, and capture the relationship between time steps; (3) feature fusion: fuse the CNN features and self-attention output. Although attention mechanisms designed for CNN and applied to various fields have been proven to be effective in time series data analysis, the unique temporal and channel-related characteristics of time series data remain a major challenge. Existing attention mechanisms are often inefficient or overly complex when combined with CNN, resulting in longer training cycles and increased computational complexity. More importantly, these attention mechanisms combined with CNN have limited generalization capabilities across application scenarios and may perform well in one task but poorly in another. This highlights the difficulty of creating a general deep learning solution for all CNN applications and the need to extend and enhance deep learning-based CNNs to achieve higher accuracy and generalization capabilities in diverse tasks. Most existing self-attention mechanisms only perform one-dimensional processing, such as the spatial self-attention proposed by Li et al. (2023). Few attention mechanisms can consider both channel and time dimensions. Classical temporal attention mechanisms (such as the work of Bahdanau et al. (2016) and Chorowski et al. (2015)) have two major flaws: first, the calculation of the attention weight for each time sample depends only on the value at that specific moment, ignoring the potential contextual information of adjacent time points; second, the temporal position of each time sample has no effect on the calculated attention weight. In addition, most time series models tend to ignore the impact of frequency information on time series tasks and cannot fully capture the inherent characteristics of time series data. There are certain defects in improving the classification accuracy. Summary of the invention
[0008] The technical problem to be solved by the present invention is to provide an industrial anomaly detection method that combines temporal feature attention and channel attention mechanism. The method combines temporal feature attention technology and channel attention mechanism, which can not only capture the key temporal features (TEAM_T) in time series data, but also highlight the discriminative feature information (TEAM_C) in different channels, thereby improving the accuracy of industrial anomaly detection.
[0009] The technical solution adopted by the present invention is to provide an industrial anomaly detection method combining temporal feature attention and channel attention mechanism, the method comprising the following steps:
[0010] S1. Obtain an industrial time series anomaly detection data set, and preprocess the industrial time series anomaly detection data set as original multivariate time series data to obtain preprocessed original multivariate time series data;
[0011] S2, dividing the preprocessed original multivariate time series data into training sample sets X train And the test sample set X test ;
[0012] S3, constructing a TEAM_DenseNet network model; the TEAM_DenseNet network model includes a Block1 module and a Block2 module; the Block1 module includes Layer0, Layer1, Layer2, Layer3 and Layer4; the Block2 module includes Layer5, Layer6 and Layer7; wherein, Layer0 is composed of a 7×7 convolutional layer with a step size of 2, and Layer1 and Layer3 have the same structure, both of which are DenseBlocks, and they both include a batch normalization (BN) layer, a ReLU activation layer, a 3×3 convolutional layer and a 2×2 average pooling layer, Layer2 is a temporal attention layer, namely TEAM_T, Layer4 is a channel attention layer, namely TEAM_C, Layer5 is a global average pooling layer, Layer6 is a linear mapping layer, and Layer7 is a probability layer;
[0013] S4, the training sample set X train Input into the TEAM_DenseNet network model for training to obtain the trained TEAM_DenseNet network model; the specific training process is:
[0014] S4.1, the training sample set X train Input the TEAM_DenseNet network model, perform convolution operation by Layer0, and then perform feature reuse operation by Layer1 to obtain the output feature map F X , specifically expressed as: F X =CNN prec (X train ); Among them, CNN prec It is a general term for Layer 0 and Layer 1;
[0015] S4.2, Layer 2 uses a one-dimensional CNN filter as a pattern detector to output the feature map F X Convert to dimension The feature map of C represents the number of feature channels; the dimension is then transformed into The feature map is projected into the query matrix Q c , key matrix K c Sum value matrix V c ,in, Calculate the query matrix Q c and key matrix K c The dot product between them, perform SoftMax operation on each dot product and sum it with the value matrix V t Perform a weighted sum and then inverse transpose it to produce a The time characteristics of Y t ;
[0016] S4.3. The time feature Y t Input Layer3 to perform feature reuse operation and obtain output feature map F' X ; Specifically expressed as: F' X =CNN rest (Y t ); Among them, CNN rest Indicates Layer3;
[0017] S4.4, in Layer4, the output feature map F' is linearly projected X Projection is the query matrix Q c , key matrix K c and the value matrix V c , Calculate the query matrix Q c and key matrix K c The dot product between them, applying the SoftMax operation to each dot product and summing the value matrix V c Perform a weighted sum and then inverse transpose it to produce a The channel attention feature Y c ;
[0018] S4.5, apply Layer5 to perform global average pooling, and transform the value x of the i-th channel i The shape from Transformed to The global average pooling is specifically expressed as follows: Among them, x i ∈Y c ;
[0019] S4.6, apply Layer6 to pool the feature vector x pooled Linearly map to the classification label space, and the output shape after pooling is Transformed to Where n represents the number of categories of industrial equipment status, which is 2. 0 and 1 are used to represent the two states of the equipment, 0 represents the normal operation state, and 1 represents the abnormal state. The Layer6 layer converts the pooled feature vector x pooled The operation of linear mapping to the classification label space is specifically expressed as: z = W x pooled +b; where W is the weight matrix and b is the bias vector with a shape of 1×n;
[0020] S4.7, convert the feature z into a probability distribution through Layer7; for each state j of the device, apply the Softmax function to calculate the corresponding probability value p j ;
[0021] S4.8, the probability value p j Use the argmax function to return the index of the maximum value in the probability distribution and get the predicted state y, which is specifically expressed as: y = argmax(p j );
[0022] S5. Test set data X test The trained TEAM_DenseNet network model is input for detection, and the trained TEAM_DenseNet network model generates an abnormal prediction state y for each sample.
[0023] The beneficial effects of the present invention are as follows: the present invention integrates time attention and channel attention mechanisms, and this dual attention mechanism enables the model to effectively capture relevant features in the time and channel dimensions, and provides a more comprehensive and robust feature extraction capability compared to a model that relies only on a single attention mechanism or a traditional CNN; the present invention can better suppress irrelevant samples and noise, and improve the robustness of the model; this makes the present invention more adaptable to variations and noise in data, thereby improving the stability and reliability of the model; the method of the present invention is applicable to various CNN architectures and has strong adaptability; the present invention can be seamlessly integrated into various existing CNN architectures and can be applied to any layer; such a design makes the present invention highly flexible in application and can be adjusted and optimized according to specific task requirements.
[0024] Preferably, in step S1, the specific process of preprocessing the industrial time series anomaly detection data set as the original multivariate time series data is as follows: the original multivariate time series data is set to contain data at T moments, the dimension of the data at each moment is d, and the multivariate time series data is represented as a T×d matrix; starting from the edge of the multivariate time series data, the data content with a window length of m is expanded outward, and the multivariate time series data is divided into a number of data blocks with the i-th moment as the center and w moments as the window size using window sliding, and the data block is recorded as Xi , where the value range of i is 1, 2, 3, …, T, and a series of time series data blocks are obtained.
[0025] Preferably, the specific process of step S4.2 is expressed as follows:
[0026] Q t =F X ·W q ,
[0027] K t =F X ·W k ,
[0028] V t =F X ·W v ,
[0029]
[0030] Among them, W q , W k ,W v are weight matrices, respectively used to transform the feature map F X Mapped to query space, key space, and value space.
[0031] Preferably, the specific process of step S4.4 is expressed as follows:
[0032] F′ X =R(F′ X ),
[0033] Q c =F′ X ·W q ,
[0034] K c =F′ X ·W k ,
[0035] V c =F′ X ·W v ,
[0036]
[0037] Among them, R(·) represents the transposition operation, R -1 (·) indicates inverse transposition.
[0038] Preferably, in step S4.7, the probability value p j Specifically expressed as: Among them, z jis the jth component of the linear mapping layer output with respect to feature z, z k is the sum of all category index scores, p j Represents the probability of normality and abnormality. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 is a flow chart of the method of the present invention;
[0040] Figure 2 The following is a comparison chart of the anomaly detection accuracy of each method under different time series classification data sets. DETAILED DESCRIPTION
[0041] The present invention is further described in detail below with reference to the accompanying drawings.
[0042] The present invention relates to an industrial anomaly detection method based on temporal feature attention and attention mechanism, the flow chart of the method is as follows Figure 1 As shown, the following steps are included:
[0043] Step 1: Data preprocessing
[0044] Obtain the industrial time series anomaly detection dataset SECOMManufacturingData2.0; the SECOMManufacturing Data2.0 dataset is mainly used for quality inspection tasks in the manufacturing process. Each sample is a time series, which describes the changes in the measurement values of multiple sensors over time. The goal of this dataset is to judge whether an instance in the production process is qualified based on the given time series data. The time series length of this dataset is 180, containing 590 features. Each instance is represented as a 590-dimensional single-step data point, which is suitable for multivariate anomaly detection tasks. The dataset contains a total of 1567 instances. Take SECOMManufacturingData2.0 as the original multivariate time series data and preprocess it to obtain the preprocessed original multivariate time series data;
[0045] The specific process of preprocessing the industrial time series anomaly detection data set as the original multivariate time series data is as follows: the original multivariate time series data is set to contain data at T moments, the dimension of the data at each moment is d, and the multivariate time series data is represented as a T×d matrix; in order to retain the integrity of the time series data, an edge filling strategy is adopted; starting from the edge of the multivariate time series data, the data content with a length of 1 window length is expanded outward, so as to ensure that the edge information of the time series data is completely retained; using window sliding to divide the multivariate time series data into several data blocks with the ith moment as the center and w moments as the window size, denoted as X i, where the value range of i is 1, 2, 3, ···, T, and a series of time series data blocks are obtained. Each block contains information about the surrounding environment, and the labels of the multivariate time series data need to be processed accordingly.
[0046] Step 2: Divide the preprocessed original multivariate time series data into training sample sets X train And the test sample set X test ;
[0047] The division rule is: randomly select n% of the original multivariate time series data obtained in step 1 as the training sample set X train The remaining part 100%-n% is used as the test sample set X test , Among them, n is the percentage value, N train =N1×n%+N2×n%+···+N C ×n%,N test =N g -N train , the value range of j is 1, 2, ···, N train , the value range of k is 1, 2, ···, N test .
[0048] Step 3: Build the TEAM_DenseNet network model
[0049] Combine the DenseNet network with the TEAM module to build the TEAM_DenseNet network model;
[0050] The TEAM_DenseNet network model includes a Block1 module and a Block2 module; the Block1 module includes Layer0, Layer1, Layer2, Layer3 and Layer4; the Block2 module includes Layer5, Layer6 and Layer7; wherein, Layer0 is composed of a 7×7 convolutional layer with a step size of 2, and Layer1 and Layer3 have the same structure, both of which are DenseBlocks, and they both include a batch normalization (BN) layer, a ReLU activation layer, a 3×3 convolutional layer and a 2×2 average pooling layer, Layer2 is a temporal attention layer, namely TEAM_T, Layer4 is a channel attention layer, namely TEAM_C, Layer5 is a global average pooling layer, Layer6 is a linear mapping layer, and Layer7 is a probability layer.
[0051] Step 4: The training sample set X trainInput into the TEAM_DenseNet model for training;
[0052] The specific training process is as follows:
[0053] S4.1, the training sample set X train Input the TEAM_DenseNet network model, perform convolution operation by Layer0, and then perform feature reuse operation by Layer1 to obtain the output feature map F X , specifically expressed as: F X =CNN prec (X train ); Among them, CNN prec It is a general term for Layer 0 and Layer 1;
[0054] S4.2, Layer 2 uses a one-dimensional CNN filter as a pattern detector to output the feature map F X Convert to dimension The feature map of C represents the number of feature channels; the dimension is then transformed into The feature map is projected into the query matrix Q c , key matrix K c Sum value matrix V c ,in, Calculate the query matrix Q c and key matrix K c The dot product between them, perform SoftMax operation on each dot product and sum it with the value matrix V t Perform a weighted sum and then inverse transpose it to produce a The time characteristics of Y t ; The specific process is expressed as:
[0055] Q t =F X ·W q ,
[0056] K t =F X ·W k ,
[0057] V t =F X ·W v ,
[0058]
[0059] Among them, W q ,W k ,W vare weight matrices, respectively used to transform the feature map F X Map to query space, key space, and value space;
[0060] S4.3. The time feature Y t Input Layer3 to perform feature reuse operation and obtain output feature map F' X ; Specifically expressed as: F' X =CNN rest (Y t ); Among them, CNN rest Indicates Layer3;
[0061] S4.4, in Layer4, the output feature map F' is linearly projected X Projection is the query matrix Q c , key matrix K c and the value matrix V c , Calculate the query matrix Q c and key matrix K c The dot product between them, applying the SoftMax operation to each dot product and summing the value matrix V c Perform a weighted sum and then inverse transpose it to produce a The channel attention feature Y c ; Perform a transposition operation to calculate features in the channel dimension. The specific process is expressed as:
[0062] F' X =R(F' X ),
[0063] Q c =F' X ·W q ,
[0064] K c =F' X ·W k ,
[0065] V c =F' X ·W v ,
[0066]
[0067] Among them, R(·) represents the transposition operation, R -1 (·) indicates reverse transposition;
[0068] S4.5, apply Layer5 to perform global average pooling, and transform the value x of the i-th channel i The shape from Transformed to The global average pooling is specifically expressed as follows: Among them, x i ∈Y c ;The purpose of using the global average pooling layer is to extract key information from the time series and reduce the amount of data;
[0069] S4.6, apply Layer6 to pool the feature vector x pooled Linearly map to the classification label space, and the output shape after pooling is Transformed to Where n represents the number of categories of industrial equipment status, which is 2. 0 and 1 are used to represent the two states of the equipment, 0 represents the normal operation state, and 1 represents the abnormal state. The Layer6 layer converts the pooled feature vector x pooled The operation of linear mapping to the classification label space is specifically expressed as: z = W x pooled +b; where W is the weight matrix and b is the bias vector with a shape of 1×2;
[0070] S4.7, convert the feature z into a probability distribution through Layer7; for each state j of the device, apply the Softmax function to calculate the corresponding probability value p j , which is expressed as: Among them, z j is the jth component of the linear mapping layer output with respect to feature z, z k is the sum of all category index scores, used for normalization so that the output forms a valid probability distribution, p j Indicates the probability of normality and abnormality;
[0071] S4.8, the probability value p j Use the argmax function to return the index of the maximum value in the probability distribution and get the predicted state y, which is specifically expressed as: y = argmax(p j ).
[0072] Step 5: Test set data X test The trained TEAM_DenseNet network model is input for detection, and the trained TEAM_DenseNet network model generates an abnormal prediction state y for each sample;
[0073] Among them, by comparing the abnormal prediction state y with the true label label on the test set, the following classification results can be obtained: 1. TN (true negative): samples correctly predicted as normal, that is, the model predicts that they are normal and the true label is also normal; 2. FP (false positive): samples that are incorrectly predicted as abnormal by the normal sample, that is, the model predicts that they are abnormal but the true label is normal; 3. FN (false negative): samples that are missed as normal by the abnormal sample, that is, the model predicts that they are normal but the true label is abnormal; the formula is as follows:
[0074] TN=∑(y=0∧label=0);
[0075] EP=∑(y=1∧label=0);
[0076] FN=∑(y=0∧label=1);
[0077] Use TN, EP, and FN to calculate the four evaluation indicators of Accuracy, Precision, Recall, and F1-Score to measure the performance of the model in industrial anomaly detection tasks. The formula is as follows:
[0078]
[0079] Figure 2 The comparison chart of anomaly detection accuracy using TEAM+DenseNet method, CBAM+DenseNet method, GC+DenseNet method, SE+DenseNet method and the method of the present invention under different time series classification data sets is shown. In the figure, the x-axis represents the number of channels, and the y-axis represents the anomaly detection accuracy. The common industrial anomaly detection data set SMAP (Soil MoistureActive Passive) was used in the experiment, and the same preprocessing was performed on each module and data set. The performance was evaluated by predicting whether the sample was abnormal in a classification manner. When the number of channels was set to 20, 40, 60, 80, 100, 120 and 140 respectively, the method of the present invention achieved the best detection performance in all configurations. The experiment used the Adam optimizer (learning rate of 0.001, batch size of 64), the training round was 200, the early stopping mechanism was used to prevent overfitting, and the loss function was the cross entropy loss. The hardware platform is NVIDIA 3060GPU (12GB video memory), equipped with Intel i7-12700CPU and 16GB memory, and the software environment is based on PyTorch 1.10 and Python 3.8, running on Windows 11. The experimental results show that the proposed method shows significant robustness and superiority in processing multi-channel time series data, and can better capture data features. As the number of channels increases, the anomaly detection performance is further improved.
Claims
1. An industrial anomaly detection method based on time and channel dual attention mechanism, characterized by: The method comprises the following steps: S1. Obtain an industrial time series anomaly detection data set, and preprocess the industrial time series anomaly detection data set as original multivariate time series data to obtain preprocessed original multivariate time series data; S2, dividing the preprocessed original multivariate time series data into training sample sets X train And the test sample set X test ; S3, construct a TEAM_DenseNet network model; the TEAM_DenseNet network model includes a Block1 module and a Block2 module; the Block1 module includes Layer0, Layer1, Layer2, Layer3 and Layer4; the Block2 module includes Layer5, Layer6 and Layer7; wherein, Layer0 is composed of a 7×7 convolutional layer with a step size of 2, and Layer1 and Layer3 have the same structure, both of which are DenseBlock, and they both include a batch normalization (BN) layer, a ReLU activation layer, a 3×3 convolutional layer and a 2×2 average pooling layer, Layer2 is a temporal attention layer, namely TEAM_T, Layer4 is a channel attention layer, namely TEAM_C, Layer5 is a global average pooling layer, Layer6 is a linear mapping layer, and Layer7 is a probability layer; S4, the training sample set X train Input into the TEAM_DenseNet network model for training to obtain the trained TEAM_DenseNet network model; the specific training process is: S4.1, the training sample set X train Input the TEAM_DenseNet network model, perform convolution operation by Layer0, and then perform feature reuse operation by Layer1 to obtain the output feature map F X , specifically expressed as: F X =CNN prec (X train ); Among them, CNN prec It is a general term for Layer 0 and Layer 1; S4.2, Layer 2 uses a one-dimensional CNN filter as a pattern detector to output the feature map F X Convert to dimension The feature map of C represents the number of feature channels; the dimension is then transformed into The feature map is projected into the query matrix Q c , key matrix K c Sum value matrix V c ,in, Calculate the query matrix Q c and key matrix K c The dot product between them, perform SoftMax operation on each dot product and sum it with the value matrix V t Perform a weighted sum and then inverse transpose it to produce a The time characteristics of Y t ; S4.3, input the time feature Yt into Layer3 layer for feature reuse operation, and obtain the output feature map F' X ; Specifically expressed as: F' X =CNN rest (Y t ); Among them, CNN rest Indicates Layer3; S4.4, in Layer4, the output feature map F' is linearly projected X Projection is the query matrix Q c , key matrix K c and the value matrix V c , Calculate the query matrix Q c and key matrix K c The dot product between them, applying the SoftMax operation to each dot product and summing the value matrix V c Perform a weighted sum and then inverse transpose it to produce a The channel attention feature Y c ; S4.5, apply Layer5 to perform global average pooling, and transform the value x of the i-th channel i The shape from Transformed to The global average pooling is specifically expressed as follows: Among them, x i ∈Y c ; S4.6, apply Layer6 to pool the feature vector x pooled Linearly map to the classification label space, and the output shape after pooling is Transformed to Where n represents the number of categories of industrial equipment status, which is 2. 0 and 1 are used to represent the two states of the equipment, 0 represents the normal operation state, and 1 represents the abnormal state. The Layer6 layer converts the pooled feature vector x pooled The operation of linear mapping to the classification label space is specifically expressed as: z = W x pooled +b; where W is the weight matrix and b is the bias vector with a shape of 1×n; S4.7, convert the feature z into a probability distribution through Layer7; for each state j of the device, apply the Softmax function to calculate the corresponding probability value p j ; S4.8, the probability value p j Use the argmax function to return the index of the maximum value in the probability distribution and get the predicted state y, which is specifically expressed as: y = argmax(p j ); S5. Test set data X test The trained TEAM_DenseNet network model is input for detection, and the trained TEAM_DenseNet network model generates an abnormal prediction state y for each sample.
2. The industrial anomaly detection method based on time and channel dual attention mechanism according to claim 1 is characterized by: In step S1, the specific process of preprocessing the industrial time series anomaly detection data set as the original multivariate time series data is as follows: the original multivariate time series data is set to contain data at T moments, the dimension of the data at each moment is d, and the multivariate time series data is represented as a T×d matrix; starting from the edge of the multivariate time series data, the data content with a window length of m is expanded outward, and the multivariate time series data is divided into a number of data blocks with the i-th moment as the center and w moments as the window size using window sliding, and the data block is recorded as X i , where the value range of i is 1, 2, 3, …, T, and a series of time series data blocks are obtained.
3. The industrial anomaly detection method based on time and channel dual attention mechanism according to claim 2 is characterized by: The specific process of step S4.2 is expressed as follows: Q t =F X ·W q , K t =F X ·W k , V t =F X ·W v , Among them, W q , W k , W v are weight matrices, respectively used to transform the feature map F X Mapped to query space, key space, and value space.
4. The industrial anomaly detection method based on time and channel dual attention mechanism according to claim 3 is characterized by: The specific process of step S4.4 is expressed as follows: F′ X =R(F′ X ), Q c =F′ X ·W q , K c =F′ X ·W k , V c =F′ X ·W v , Among them, R(·) represents the transposition operation, R -1 (·) indicates inverse transposition.
5. The industrial anomaly detection method based on time and channel dual attention mechanism according to claim 4 is characterized in that: In step S4.7, the probability value p j Specifically expressed as: Among them, z j is the jth component of the linear mapping layer output with respect to feature z, z k is the sum of all category index scores, p j Represents the probability of normality and abnormality.