A robust unsupervised anomaly detection method based on stacked width learning system

Abstract

The present invention provides a robust unsupervised anomaly detection method based on a stack width learning system (BLS). The method comprises the following steps: constructing an objective function for the robust stack width learning system; calculating the output weights of the robust stack width learning system via ridge regression; determining the predicted output of each robust stack width learning system submodule; calculating sample weight factors based on a kernel density estimation function, and identifying outliers based on the residual probability distribution density of the samples. By fully leveraging the residual characteristics of the stack width learning system (BLS) and assigning different weighting factors based on the discreteness of the sample points, the method eliminates interference from outliers and noise on detection. Furthermore, the robust stack width learning system (BLS) can adaptively process unlabeled data. Its dynamic stack structure and incremental learning algorithm improve both robustness and generalization capabilities while enhancing detection accuracy.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent learning, and in particular relates to a robust unsupervised anomaly detection method based on a stack width learning system. Background Art

[0002] Anomaly detection has always been a hot topic in various fields, resulting in the emergence of traditional anomaly detection algorithms tailored to the needs of different fields. However, compared to machine learning algorithms, traditional anomaly detection methods perform poorly in processing high-dimensional, massive datasets. They are unable to achieve high detection accuracy when faced with high computational complexity, multiple types of anomalies, and nonlinear coupling relationships between variables.

[0003] Anomaly detection refers to the discovery of outliers, anomalies, faults, defects, or errors in collected data. Anomalies are mainly divided into three categories: point anomalies, continuous anomalies, and group anomalies. Since the label determines whether a sample is normal or abnormal, anomaly detection is mainly divided into three categories: supervised anomaly detection (SAD), semi-supervised anomaly detection (SSAD), and unsupervised anomaly detection (UAD). Among them, both SAD and SSAD methods have certain limitations and require labeled training data. Compared with UAD, SAD and SSAD have higher requirements for sample labels and involve training classifiers. UAD is mainly used to discover instances that do not match other data. Specifically, in actual industrial application scenarios, data sets are collected in real time and new data will be continuously generated, which limits the generation of corresponding labels for the training data set. In addition, creating labels itself is a costly and time-consuming process, especially when large data sets are involved.

[0004] In recent years, deep learning has been widely used in anomaly detection due to its deep network structure, powerful feature extraction capabilities, strong approximate learning ability, and low error after fitting complex models. Deep learning has significantly outperformed traditional methods in anomaly detection when solving challenging real-world problems. Existing research shows that generative models, self-supervised learning models, and deep single-class classification models are three common deep learning methods used in unsupervised image detection (UAD). When applying generative model-based methods to anomaly detection, two common models are used, depending on their backbone networks: autoencoders and generative adversarial networks (GANs). Autoencoders are a widely used basic architecture in UAD, paving the way for numerous improved methods. GANs, due to their structural characteristics, can handle the problem of insufficient labeled data and are also a common method in UAD. While self-supervised learning methods have achieved significant progress in the accuracy and speed of representation learning and image anomaly detection in recent years, they still have several limitations. Firstly, they require a large amount of sample data. Due to the large number of parameters and complex network structures, this limited sample data can lead to overfitting and poor performance. On the other hand, during deep learning training, adjusting the network's connection weights layer by layer is time-consuming, consumes significant computing resources, and leads to higher hardware costs. The emergence of the breadth-based learning system (BLS) has significantly improved these shortcomings. First proposed in 2018, BLS enriches the scope of machine learning and expands the form of network neuron architecture. It consists of input data, a feature layer, an enhancement layer, and an output. Input data is extracted through feature nodes, which are then augmented. Enhancement nodes are extensions of these feature nodes, and the enhancement matrix is ​​passed to the output layer. Incremental learning is the core of BLS, which offers the advantage of eliminating the need for recalculation when faced with increasing data. Furthermore, BLS can efficiently train and update models, using a simple linear structure to reduce computational complexity and workload, and easily handles small batches of data. Its flat, simple linear structure makes BLS easy to reshape and integrate, and has found widespread application in many fields. However, basic BLS requires supervised training on labeled data. In industrial manufacturing, data acquisition is real-time, making it difficult to extract corresponding labels from the source data. Furthermore, obtaining labels is expensive and time-consuming. Therefore, the performance of anomaly detection using the BLS basic structure will be limited.

[0005] To address the performance shortcomings of BLS in the unlabeled case, this paper proposes stacked BLS, which uses stacking rules and residuals to superimpose BLS blocks. The residuals generated by the previous BLS block are then propagated to the next BLS block, ultimately forming the output of all BLS blocks. Deeper stacking can also be performed as needed. Considering that in many industrial scenarios, actual production data collected is often contaminated by noise and outliers due to random events, sensor failures, human interference, and environmental noise, affecting detection accuracy. Summary of the Invention

[0006] In view of this, the purpose of the present invention is to provide the following technical solutions:

[0007] A robust unsupervised anomaly detection method based on a stack width learning system comprises the following steps:

[0008] Objective function for constructing a robust stack width learning system:

[0009] calculating output weights of the robust stack width learning system by ridge regression;

[0010] determining a predicted output of each robust stack width learning system submodule;

[0011] Based on the kernel density estimation function, the weight factor of the sample is calculated, and outliers are identified according to the residual probability distribution density of the sample.

[0012] Preferably, calculating the output weight of the robust stack width learning system by ridge regression specifically includes:

[0013] Get approximate results for ridge regression;

[0014] The output weight is calculated based on the approximate result, and the calculation process is as follows:

[0015] The expression of the approximate result of the ridge regression is:

[0016]

[0017] Simplify the above expression: Enter

[0018] get:

[0019] Among them, A represents the augmented matrix composed of feature nodes and enhancement nodes; A + represents the pseudo-inverse of matrix A; A T represents the transpose of matrix A; I represents the identity matrix; V i Represents the mapping matrix of feature nodes; W i represents the output weight of the width learning system; WZ Represents the corresponding weight of the feature node matrix; W H represents the corresponding weight of the enhanced node matrix; θ represents the weight factor.

[0020] The robust stack width learning system includes N robust stack width learning system submodules;

[0021] The prediction output expression of the robust stack width learning system submodule is:

[0022]

[0023] in,

[0024]

[0025] Where, The augmented matrix representing the feature nodes and enhancement nodes of the robust stack width learning system submodule; Represents the augmented node matrix of the robust stack width learning system submodule.

[0026] Preferably, calculating the weight factor θ of the sample specifically includes the following steps:

[0027] Obtain a training sample set;

[0028] Calculating sample residuals in the training sample set;

[0029] Obtain the residual probability density function f(x) based on the kernel density estimation function;

[0030] Calculate the weight factor θ of the i-th sample point according to the residual probability density function i ;

[0031] Based on θ i Construct the weight matrix θ;

[0032] The samples in the training sample set are unlabeled input data.

[0033] Preferably, identifying outliers based on the residual probability distribution density of the sample specifically includes:

[0034] Obtaining judgment factors for identifying outliers;

[0035] Based on the above judgment factors, the identification of abnormal values ​​is judged:

[0036] Determine whether MAX is lower than the preset value ε. If yes, the iterative process stops; if no, the iterative process continues until the result is yes; Determine whether l reaches the maximum number of iterations. If yes, the iterative process stops; if no, the iterative process continues until the result is yes;

[0037] Where l represents the lth iteration; MAX represents the difference between the maximum absolute values ​​of two consecutive output weights; and ε represents the preset threshold.

[0038] Preferably, the MAX expression is:

[0039]

[0040] in, represents the output weight of the i-th subsystem obtained in the l-th iteration.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] (1) Adding robustness on the basis of stacked BLS not only fully utilizes the deep structure, residual training characteristics and adaptability of stacked BLS, but also improves the robustness and stability of the UAD method, thereby relaxing the need for data labels and making it more convenient to process real scene data.

[0043] (2) The weight factor is adaptively assigned according to the distribution of each sample, which enhances the contribution of normal sample data to the detection model and eliminates the interference of noise and outliers in the sample.

[0044] (3) Robust stacked BLS performs well in unsupervised anomaly detection. Robust stacked BLS outperforms typical anomaly detection learning methods on the MVTecAD dataset with good accuracy and efficiency.

[0045] (4) Robust stacked BLS has achieved impressive results using the most mature deep anomaly detection methods and leads in some performance indicators. In addition, it retains the advantages of fast computation speed and strong interpretability. It can quickly solve the pseudo-inverse through ridge regression, significantly improving learning efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a flowchart of the method of the present invention.

[0047] Figure 2 Schematic diagram of the test situation of an embodiment of the present invention.

[0048] Figure 3 Schematic diagram of the ROC curve used in the embodiment of the present invention.

[0049] Figure 4 Schematic diagram of the PR curve used in the embodiment of the present invention.

[0050] Figure 5 Schematic diagram of the proportion of normal and abnormal samples correctly classified by robust stacked BLS in image classification. DETAILED DESCRIPTION

[0051] The technical solutions provided by the present invention are described in detail below with reference to the embodiments, but they should not be construed as limiting the scope of protection of the present invention.

[0052] Example 1:

[0053] This embodiment discloses a robust unsupervised anomaly detection method based on a stack width learning system, comprising the following steps:

[0054] The objective function for constructing a robust stack width learning system is:

[0055]

[0056] The weight factor of the sample is integrated into the robust stacked BLS objective function and is calculated as follows:

[0057] Among them, e i Represents actual output and prediction

[0058] ste i =U i -Y i

[0059] The residual between the outputs, λ represents the regularization parameter to prevent overfitting of the model and improve generalization.

[0060] Compute the output weights of the robust stack width learning system via ridge regression;

[0061] determining a predicted output of each robust stack width learning system submodule;

[0062] Based on the kernel density estimation function, the weight factor of the sample is calculated, and the outliers are identified according to the residual probability distribution density of the sample;

[0063] Among them, U i represents the predicted output of the i-th BLS block; Y i Indicates the actual output; represents the randomly generated weight matrix associated with the feature node in the i-th BLS block; represents the weight matrix associated with the randomly generated enhanced node in the i-th BLS block; λ represents the regularization parameter, 1≤i≤N.

[0064] Specifically:

[0065] In this embodiment, the output weight of the robust stack width learning system is calculated by ridge regression, specifically including:

[0066] Get approximate results for ridge regression;

[0067] The output weight is calculated based on the approximate result. The calculation process is as follows:

[0068] The expression of the approximate result of ridge regression is:

[0069]

[0070] Simplify the above expression: Enter

[0071] get:

[0072] Among them, A represents the augmented matrix composed of feature nodes and enhancement nodes; A + represents the pseudo-inverse of matrix A; A T represents the transpose of matrix A; I represents the identity matrix; V i Represents the mapping matrix of feature nodes; W i represents the output weight of the width learning system; W Z Represents the corresponding weight of the feature node matrix; W H represents the corresponding weight of the enhanced node matrix; θ represents the weight factor.

[0073] In this embodiment, the robust stack width learning system includes N robust stack width learning system submodules;

[0074] The prediction output expression of the robust stack width learning system submodule is:

[0075]

[0076] in,

[0077]

[0078] Where, The augmented matrix representing the feature nodes and enhancement nodes of the robust stack width learning system submodule; Represents the augmented node matrix of the robust stack width learning system submodule.

[0079] In this embodiment, calculating the sample weight factor θ specifically includes the following steps:

[0080] A training set of samples is obtained. Specifically, in this example, the training set is extracted from the MVTec dataset, which includes 15 categories of images. The training set contains 3,629 defect-free images, and the test set contains 1,725 ​​images, including images with no anomalies and a variety of defect types. In addition, five categories contain different texture types: carpet, net, leather, tile, and wood. The remaining ten categories represent different types of objects and can be divided into two categories: objects with fixed shapes, such as bottles and metal nuts, and objects with deformable appearances, such as cables and hazelnuts. Some objects are captured in roughly aligned arrangements, such as toothbrushes, capsules, and pills, while others are captured in random positions, such as metal nuts, screws, and hazelnuts. The test set contains 73 different types of defects, generated by simulating real-world industrial inspection scenarios. These defects include structural defects such as object deformation or missing parts, and surface defects such as scratches or dents. Furthermore, in this example, all images have a resolution between 700*700 and 1024*1024 pixels and are cropped to an appropriate output size. These images were acquired under highly controlled lighting conditions, where three object categories (grid, spiral, and zipper) are only available as single-channel images.

[0081] Calculate the sample residual in the training sample set; sample residual R i The expression is:

[0082] Among them, R i =[r1,r2,...,r N ] T and r i (i=1,...,N) represents the residual of the i-th sample;

[0083] The residual probability density function f(x) is obtained based on the kernel density estimation function; the expression of the residual probability density function f(x) is:

[0084] Among them, τ represents the width of the estimation window, and represents the standard deviation of the residual, N represents the number of samples, and k(·) represents the Gaussian kernel function. The expression is:

[0085]

[0086] Calculate the weight factor θ of the i-th sample point according to the residual probability density function i ; where θ i The expression is: i =f(r i );

[0087] Based on θi Construct the weight matrix θ: θ=[θ1,...,θ N ] T ;

[0088] Among them, the samples in the training sample set are unlabeled input data.

[0089] In this embodiment, identifying outliers based on the residual probability distribution density of the sample specifically includes:

[0090] Obtaining judgment factors for identifying outliers;

[0091] Identify outliers based on the following judgment factors:

[0092] Determine whether MAX is lower than the preset value ε. If yes, the iterative process stops; if no, the iterative process continues until the result is yes; Determine whether l reaches the maximum number of iterations. If yes, the iterative process stops; if no, the iterative process continues until the result is yes;

[0093] Where l represents the lth iteration; MAX represents the difference between the maximum absolute values ​​of two consecutive output weights; and ε represents the preset threshold.

[0094] In this embodiment, the MAX expression is:

[0095]

[0096] in, represents the output weight of the i-th network obtained in the l-th iteration.

[0097] In actual implementation, in this embodiment, n images are used. n}Test set, for each test image I i , S i (p)∈R represents the true value anomaly score of each pixel,

[0098] G i (p)∈{0,1} represents the reference standard for pixel accuracy. The larger the anomaly score, the greater the possibility that the pixel is abnormal. In the anomaly detection process, t∈R is selected as the basis for predicting whether it is abnormal. Therefore, when S i (p)>t, the image pixel is predicted to be abnormal. i (p)=1, indicating that an abnormality exists. When G i (p) = 0, indicating no abnormality.

[0099] To evaluate the performance of the robust stacked BLS anomaly detection method, for each test set image, there is Figure 2The relative score of each pixel is calculated as follows: True Positive Rate (TPR) = TP / (TP+FN), False Positive Rate (FPR) = FP / (FP+TN), and Precision Rate (PRC) = TP / (TP+FP).

[0100] All of the above indicators depend on the selected threshold t, but choosing an appropriate threshold is a challenging problem in practice. Therefore, the above indicators are usually evaluated at multiple different thresholds to avoid failure in threshold selection. In addition, when observing multiple indicators at the same time, there is more reference value. Here, this embodiment selects two commonly used measurement curves and calculates the area under the curve (AU). Measurement curves: receiver operating characteristic curve (ROC) and precision recall curve (PR). ROC plots FPR and TPR, and PR plots TPR and PRC.

[0101] The ROC curve and PR curve of the robust stacked BLS in 15 types of image anomaly detection are shown as follows: Figure 3 and Figure 4 As shown. Obviously, Figure 3 The curve is close to the upper left corner, and Figure 4 Curves closer to the upper right corner indicate that a larger area under the curve indicates better method performance. The ROC and PR curves for bottles and toothbrushes are closest to ideal values, with larger areas under the curves. Furthermore, the robust stacked BLS algorithm has the highest average AU-ROC (area under the ROC curve) value among all image anomaly detection methods, and outperforms several other deep learning methods in terms of AU-PR (area under the PR curve). Figure 5 The ratio of correctly classified samples of normal and abnormal samples in image classification by robust stacked BLS is shown in Figure 2. This shows that the correct classification performance of this method is better than that of other methods.

[0102] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A robust unsupervised anomaly detection method based on stack width learning system, characterized in that The method is applied to the MVTec dataset and includes the following steps: Construct the objective function of the robust stack width learning system; The weight factor of the sample is integrated into the robust stacked BLS objective function and is calculated as follows: Among them, e i represents the residual between the actual output and the predicted output, λ represents the regularization parameter, and U i represents the predicted output of the i-th BLS block; Y i Indicates the actual output, 1≤i≤N; calculating output weights of the robust stack width learning system by ridge regression; Calculating the output weights of the robust stack width learning system by ridge regression specifically includes: Get approximate results for ridge regression; The output weight is calculated based on the approximate result, and the calculation process is as follows: The expression of the approximate result of the ridge regression is: Simplify the above expression: Enter get: Among them, A represents the augmented matrix composed of feature nodes and enhancement nodes; A + represents the pseudo-inverse of matrix A; A T represents the transpose of matrix A; I represents the identity matrix; V i Represents the mapping matrix of feature nodes; W i represents the output weight of the width learning system; W Z Represents the corresponding weight of the feature node matrix; W H represents the corresponding weight of the enhanced node matrix; θ represents the weight factor; determining a predicted output of each robust stack width learning system submodule; Based on the kernel density estimation function, the weight factor of the sample is calculated, and the outliers are identified according to the residual probability distribution density of the sample; Calculating the sample weight factor θ specifically includes the following steps: Obtain a training sample set; Calculating sample residuals in the training sample set; Obtain the residual probability density function f(x) based on the kernel density estimation function; Calculate the weight factor θ of the i-th sample point according to the residual probability density function i ; Based on θ i Construct the weight matrix θ; Wherein, the samples in the training sample set are unlabeled input data; Identifying outliers based on the residual probability distribution density of the sample specifically includes: Obtaining judgment factors for identifying outliers; Based on the above judgment factors, the identification of abnormal values ​​is judged: Determine whether MAX is lower than the preset value ε. If yes, the iterative process stops; if no, the iterative process continues until the result is yes; Determine whether l reaches the maximum number of iterations. If yes, the iterative process stops; if no, the iterative process continues until the result is yes; Where l represents the lth iteration; MAX represents the difference between the maximum absolute values ​​of two consecutive output weights; and ε represents the preset threshold.

2. The robust unsupervised anomaly detection method based on stack width learning system according to claim 1, characterized in that: The robust stack width learning system includes N robust stack width learning system submodules; The prediction output expression of the robust stack width learning system submodule is: in, Where, The augmented matrix representing the feature nodes and enhancement nodes of the robust stack width learning system submodule; Represents the augmented node matrix of the robust stack width learning system submodule.

3. The robust unsupervised anomaly detection method based on stack width learning system according to claim 1, characterized in that: The MAX expression is: in, represents the output weight of the i-th subsystem obtained in the l-th iteration.

Images