Anomaly detection system using a student-teacher network and method thereof

Abstract

The present invention relates to an anomaly detection system based on a Student-Teacher network. The anomaly detection system comprises: a learning system that trains a Student-Teacher network using normal data and stores bottleneck embedding feature vectors generated for the normal data during the training process in a memory bank; and an inference system that detects anomalies in input data using the trained Student-Teacher network and the bottleneck embedding feature vectors stored in the memory bank. The anomaly detection system according to the present invention learns an encoder and decoder-based Teacher-Student network to generate low-dimensional embedding feature vectors for normal data and stores them in a memory bank, and during inference, calculates the distance between the embedding feature vectors for the input data and the embedding feature vectors in the memory bank, thereby enabling efficient use of the memory bank and a reduction in inference speed.

Description

Technology Field

[0001] The present invention relates to an anomaly detection system and method, and more specifically, to an anomaly detection system and method based on a student-teacher network, characterized by training a Teacher-Student network with an encoder-decoder structure, generating bottleneck embedding feature vectors for normal data during the training process and storing them in a memory bank, and, during inference, configuring the system to rapidly perform anomaly detection on data using the trained Teacher-Student network and the bottleneck embedding feature vectors stored in the memory bank. Background Technology

[0002] Defect detection identifies data as either normal or abnormal based on whether it follows general patterns. Anomaly detection is not only widely used in real-world industries but is also a highly required element in the manufacturing sector; examples include product quality control, medical diagnostics, and video surveillance. Since abnormal data does not occur frequently in industry, there is a scarcity of data available. Consequently, it is difficult to solve the problem using supervised learning, and data labeling is also challenging. Therefore, due to these issues, anomaly detection is approached through unsupervised learning rather than supervised learning.

[0003] Generally, data are constructed in the form of a One-Class Classification model, where the training dataset consists only of normal data and the test dataset includes both normal and abnormal data. This learning method is called "unsupervised learning." When trained in this way, the network learns only the features of normal data, and when inferring with the test dataset after training, it becomes possible to filter out features different from the learned normal features.

[0004] Autoencoders and Generative Adversarial Networks (GANs), which are unsupervised learning methods, have been extensively studied for anomaly detection. Methods of detecting anomalies through reconstruction using generative models such as autoencoders and GANs rely on the assumption that a model trained solely on normal data can only reconstruct normal parts and cannot reconstruct abnormal parts. Figure 1 is a configuration diagram illustrating an autoencoder-based anomaly detection structure according to conventional technology. When training with normal data, the difference between the original input image and the newly generated image through the network will be small when normal data is input during inference, while the difference between the original input image and the newly generated image will be relatively large when abnormal data is input. Conventional anomaly detection methods using autoencoders and generative models can determine the presence or absence of anomalies in the data based on these difference values. However, such reconstruction methods are difficult to reconstruct when the variance between classes is large. This is because it is difficult for a single network to be trained on manufactured products of various categories.

[0005] To address these problems, many feature matching-based methods have been proposed. Feature matching methods detect defects by calculating the distance between feature distributions.

[0006] Meanwhile, anomaly detection methods utilizing pre-trained models have also recently been proposed. Recent research has demonstrated that networks pre-trained on large datasets, such as ImageNet, can extract good discriminative feature vectors for anomaly detection. Abnormal data can be identified using feature vectors of normal data extracted from pre-trained networks. For example, methods include extracting feature vectors from a pre-trained model, storing them in a memory bank, and using a KNN-based anomaly detection method, or detecting abnormal data by utilizing the distribution of normal data extracted from the pre-trained network and measuring the Mahalanobis distance. However, these methods have the disadvantage of being computationally inefficient.

[0007] Meanwhile, the Student-Teacher Feature Matching for anomaly detection (STFPM) method is an anomaly detection method utilizing a Student-Teacher network structure. It uses a pre-trained network as the Teacher Network and transfers knowledge received from the Teacher Network to a Student Network with the same structure as the Teacher Network. Figure 2 is a configuration diagram illustrating the Student-Teacher network structure according to the conventional STFPM method. This defect detection method based on the Student-Teacher network structure uses feature vectors extracted through a pre-trained network to detect defects through feature matching, and has shown superior performance compared to reconstruction-based models using conventional autoencoders and generative models. However, this method has limitations, such as requiring a large amount of memory space to store feature vectors for feature matching and having a slow inference speed for feature matching. Prior art literature

[0008] Korean Registered Patent Publication No. 10-2348536, Korean Published Patent Publication No. 10-2022-0055970, Korean Registered Patent Publication No. 10-2232138, Korean Registered Patent Publication No. 10-2476300 The problem to be solved

[0009] To solve the aforementioned problems, the present invention aims to provide an anomaly detection system and method that can efficiently use the memory bank and shorten the inference speed by training an encoder and decoder-based Teacher-Student network to extract low-dimensional embedding feature vectors for normal data and store them in a memory bank, and calculating the distance between the embedding feature vectors for the input data and the embedding feature vectors in the memory bank during inference. means of solving the problem

[0010] A Student-Teacher network-based anomaly detection system according to the first feature of the present invention for achieving the aforementioned technical problem comprises: a memory bank; a learning system that trains a Student-Teacher network using normal data and stores bottleneck embedding feature vectors in the memory bank; and an inference system that detects anomalies in input data using the pre-trained Teacher network of the Student-Teacher network and the bottleneck embedding feature vectors stored in the memory bank.

[0011] In an anomaly detection system based on a Student-Teacher network according to the first feature described above, the learning system preferably comprises: a network learning module that learns a Student-Teacher network using normal data and generates a learned bottleneck embedding feature vector that encapsulates the features of the normal data into a low-dimensional space; and a subsampling module that subsamples the bottleneck embedding feature vectors for the normal data generated during the learning process and stores the subsampled bottleneck embedding feature vectors in a memory bank.

[0012] In an anomaly detection system based on a Student-Teacher network according to the first feature described above, the Student-Teacher network preferably comprises: a Teacher network that is pre-trained to extract feature vectors for data and has fixed weights; a feature vector linking layer that combines feature vectors generated from the Teacher network; an embedding layer that is trained to condense multi-scale feature vectors of data into a low-dimensional space containing such vectors and extracts embedding feature vectors from the combined feature vectors; and a Student network that is trained to reconstruct data from the embedding feature vectors of the embedding layer.

[0013] In a Student-Teacher network-based anomaly detection system according to the first feature described above, it is preferable that the Teacher network is composed of an encoder structure and the Student network is composed of a decoder structure.

[0014] In a Student-Teacher network-based anomaly detection system according to the first feature described above, the inference system is equipped with an anomaly detection module, and when new data is input, the anomaly detection module extracts an embedding feature vector through a pre-trained Teacher Network of the Student-Teacher network, calculates the distance between the extracted feature vector and the embedding feature vector stored in a memory bank to calculate an outlier score, and preferably identifies the location of an outlier through the outlier score.

[0015] An anomaly detection method according to the second feature of the present invention relates to an anomaly detection method in an anomaly detection system equipped with a memory bank and a Student-Teacher network, comprising: (a) a step of training a Student-Teacher network using normal data; (b) a step of generating a bottleneck embedding feature vector trained to condense the features of normal data into a low-dimensional space during the training process of the Student-Teacher network; (c) a step of storing the generated bottleneck embedding feature vectors in a memory bank; and (d) a step of detecting anomalies in input data using the pre-trained Teacher Network of the Student-Teacher network and the bottleneck embedding feature vectors stored in the memory bank.

[0016] In the Student-Teacher network-based anomaly detection method according to the second feature described above, the step (c) preferably involves subsampling bottleneck embedding feature vectors generated during the learning process and storing the subsampled bottleneck embedding feature vectors in a memory bank.

[0017] In the Student-Teacher network-based anomaly detection method according to the second feature described above, it is preferable that the Student-Teacher network extracts feature vectors for data through a Teacher network pre-trained to extract feature vectors for data, combines feature vectors generated from the Teacher network, and trains the network to imply a low-dimensional space containing multi-scale feature vectors of data, thereby extracting low-dimensional embedding feature vectors from the combined feature vectors, and trains the Student network to reconstruct data from the embedding feature vectors.

[0018] In the Student-Teacher network-based anomaly detection method according to the second feature described above, it is preferable that the Teacher network is composed of an encoder structure and the Student network is composed of a decoder structure.

[0019] In the Student-Teacher network-based anomaly detection method according to the second feature described above, the step (d) preferably comprises: (d1) a step of extracting an embedding feature vector through a learned Student-Teacher network when new data is input; (d2) a step of calculating an outlier score by calculating the distance between the extracted feature vector and the feature vector stored in a memory bank; and (d3) a step of identifying the location of an outlier through the outlier score. Effects of the invention

[0020] An anomaly detection system according to the present invention learns an encoder and decoder-based Teacher-Student network to generate low-dimensional embedding feature vectors for normal data and stores them in a memory bank, and during inference, calculates the distance between the embedding feature vector for the input data and the embedding feature vectors in the memory bank. Accordingly, the anomaly detection system according to the present invention utilizes a bottleneck embedding feature vector that encapsulates the characteristics of normal data through learning, thereby enabling more efficient memory storage and shortening the inference speed. Brief explanation of the drawing

[0021] Figure 1 is a configuration diagram illustrating an autoencoder-based anomaly detection structure according to conventional technology. Figure 2 is a configuration diagram illustrating a Student-Teacher network structure according to the STFPM method according to conventional technology. FIG. 3 is a configuration diagram illustrating an anomaly detection system based on a Student-Teacher network according to a preferred embodiment of the present invention. FIG. 4 is a configuration diagram illustrating the structure of a Student-Teacher network learning module in an anomaly detection system according to a preferred embodiment of the present invention. FIG. 5 illustrates a core set memory bank algorithm in an anomaly detection system according to a preferred embodiment of the present invention. Specific details for implementing the invention

[0022] Hereinafter, an anomaly detection system and an anomaly detection method based on a Student-Teacher network according to a preferred embodiment of the present invention will be described in detail with reference to the attached drawings.

[0023] FIG. 3 is a configuration diagram illustrating an anomaly detection system based on a Student-Teacher network according to a preferred embodiment of the present invention. Referring to FIG. 3, the anomaly detection system (1) according to the present invention consists of a memory bank (2), a learning system (3), and an inference system (4). The learning system (3) trains the Student-Teacher network using normal data, subsamples bottleneck embedding feature vectors generated during the learning process, and stores the subsampled bottleneck embedding feature vectors in the memory bank (2). When input data is given, the inference system (4) extracts embedding feature vectors through the pre-trained Teacher Network of the Student-Teacher network, calculates an outlier score by calculating the Euclidean distance between the extracted embedding feature vectors and the bottleneck embedding feature vectors in the memory bank (2), and identifies the location of the anomaly using the outlier score.

[0024] Hereinafter, the learning system (3) of the anomaly detection system according to the present invention will be described in detail. The learning system comprises a network learning module (20) for training a Student-Teacher network and a subsampling module (22) for subsampling bottleneck embedding feature vectors generated during the learning process and storing them in a memory bank. Bottleneck embedding feature vectors for subsampled normal data are stored in the memory bank (2).

[0025] An anomaly detection system according to the present invention utilizes a Student-Teacher network and learns the features of normal data through a learning process of a hierarchical feature matching method. FIG. 4 is a configuration diagram illustrating the structure of a Student-Teacher network learning module in an anomaly detection system according to a preferred embodiment of the present invention. Referring to FIG. 4, the Student-Teacher network learning module (20) uses an encoder-decoder type Student-Teacher network to learn the feature distribution of normal data and comprises a Teacher Network (200) of a pre-trained encoder structure, a feature vector connection layer (210), a learnable embedding layer (220), and a Student Network (230) of a learnable decoder structure.

[0026] The Teacher Network (200) is composed of an encoder structure and is pre-trained to extract feature vectors for data, characterized by having fixed weights. The feature vector concatenation layer (210) combines the feature vectors extracted from the pre-trained Teacher Network by matching the spatial resolution size through a convolution layer and concatenating them based on channels. The embedding layer (220) utilizes the Residual Block used in the WideResNet-50 model to train the combined feature vectors so that the multi-scale feature vectors of the data are condensed into a low-dimensional space, thereby generating low-dimensional bottleneck embedding feature vectors. The Student Network (230) is trained to reconstruct data from the embedding feature vectors of the embedding layer.

[0027] Generally, unlike existing Student-Teacher networks where the Teacher Network and Student Network use the same structure, the Teacher Network and Student Network according to the present invention are characterized by following structures in the form of an encoder and a decoder, respectively. The Teacher Network according to the present invention is composed in the form of an encoder to learn an embedding feature vector that encapsulates the features of normal data, and the Student Network according to the present invention is composed in the form of a decoder to reconstruct normal data from the encapsulated embedding feature vector. At this time, the Teacher Network is a pre-trained network that is fixed so that the gradient is not updated. The encapsulated embedding feature vector in this way is referred to as a bottleneck. By encoding into a smaller-dimensional embedding feature vector, redundant information in normal data is eliminated, and only important information can be utilized in the compressed vector.

[0028] The Student-Teacher network having the aforementioned structure extracts multi-scale feature representations from an encoder-type Teacher network when input data is given, and trains a decoder-type Student network so that the feature representations of normal data can be well stored in the intermediate bottleneck embedding layer. This is a method of knowledge distillation in the Student network by targeting the multi-scale feature vectors extracted from the Teacher network. Through this training process, the bottleneck embedding layer is consequently compressed into a low-dimensional space containing the multi-scale feature vectors of normal data. In this way, the low-dimensional embedding feature vectors learned through the Student-Teacher network training module are subsampled through the subsampling module described later to reduce redundant features, and then stored in a memory bank.

[0029] The anomaly detection system according to the present invention can reduce bias in ImageNet data by utilizing mid-level features instead of high-level features. Conventional PatchCore models utilize feature vectors extracted from a pre-trained model without a separate training process, resulting in high memory usage and slow inference speeds. However, the anomaly detection system according to the present invention utilizes bottleneck embedding feature vectors that encapsulate the characteristics of normal data through training, thereby enabling more efficient memory storage and reduced inference speeds.

[0030] Below, the process of the above network learning module training the Student-Teacher network is explained in detail.

[0031] The above network training module aims to obtain bottleneck embedding feature vectors that effectively encapsulate the characteristics of normal data during the training process. A normal dataset without abnormal parts is D = {I1, I2, ... ,I n Let's call it}, and the goal is to define the phase space (Manifold) of the normal data for the teacher network and the student network. It is learned through feature matching of feature vectors extracted from layers. The input image is It is expressed as such, where w is width, h is height, and d is dimension. And It is assumed that is the order of the respective layer blocks of the teacher and student networks. The teacher network's In the case of the feature map of the i-th layer And, of the student network In the case of the feature map of the i-th layer w1, h1, and d1 are the width, height, and dimension of the corresponding layer feature map, respectively.

[0032] and and are feature vectors of the teacher network and student network, respectively, and the cosine similarity between the feature vectors extracted from the two networks is calculated. Therefore, the loss function is defined as the cosine similarity between l2-normalized feature vectors, Equation 1 is the cosine similarity, and the equation of the loss function is Equation 2.

[0033]

[0034]

[0035] Here ε is the number of feature layers. The loss value is between 0 and 1, and a lower value indicates higher cosine similarity between feature vectors. The loss value of the entire image set is equal to the average of each spatial position, and the total loss value is the sum of all values ​​corresponding to the number of feature layers.

[0036] When extracting feature vectors from a pre-trained Teacher network, duplicate feature vectors may exist within the corresponding feature map because it stores large and rich feature representatives. Therefore, the embedding layer reduces redundancy by encoding only the important features among these representatives and learns an embedding feature vector composed solely of important features that effectively represent normal data. The feature vector concatenation layer receives feature vectors extracted from the pre-trained Teacher network as input. It then uses a convolution layer to adjust the spatial resolution of the input feature vectors and performs concatenation, thereby combining the feature vectors based on channels and outputting them to the embedding layer. The embedding layer receives the combined feature vectors from the feature concatenation layer and utilizes the Residual block used in the WideResNet-50 model to condense the input feature vectors into smaller-dimensional feature vectors that effectively represent the characteristics of normal data, thereby extracting a bottleneck embedding feature vector.

[0037] The above subsampling module subsamples bottleneck embedding feature vectors extracted through the trained Student-Teacher network and stores the subsampled bottleneck embedding feature vectors in a memory bank.

[0038] The following explains the subsampling process performed by the subsampling module. First, the subsampling module projects the embedding feature vector extracted by the embedding layer onto a smaller-dimensional vector through random linear projection. d * When < d It can be represented as such. And, according to a preset ratio (%) of the total number of data, subsampling is performed from the original data by that ratio. FIG. 5 illustrates a core set memory bank algorithm in an anomaly detection system according to a preferred embodiment of the present invention. As described above, bottleneck embedding feature vectors subsampled in the memory bank by the subsampling module are stored.

[0039] Hereinafter, the inference system (4) of the anomaly detection system according to the present invention will be described in detail. The inference system comprises an anomaly detection module that detects anomalies in data using bottleneck embedding feature vectors for normal data stored in the learned Student-Teacher network and memory bank.

[0040] Hereinafter, the operation of the anomaly detection module of the inference system (4) is described in detail. When new data is input, the anomaly detection module extracts an embedding feature vector through a Teacher network. The extracted feature vector is The outlier score is calculated by determining the distance between the feature vectors stored in the memory bank. The outlier score at the image level is It is determined as the largest value among the calculated distances. The pixel-based outlier score, which is based on the image object location identification method, is calculated in a similar manner. The extracted feature vector is It is similar to h×w patches, each having an outlier score, and this size is adjusted to the original image size through bilinear interpolation. Through this process, an outlier score corresponding to each pixel can be obtained, and the location of the outlier is identified using the corresponding outlier score.

[0041] Although the present invention has been described above with reference to preferred embodiments, this is merely illustrative and does not limit the invention. Those skilled in the art will understand that various modifications and applications not exemplified above are possible within the scope of the essential characteristics of the invention. Furthermore, differences related to such modifications and applications should be interpreted as being included within the scope of the invention as defined in the appended claims. Explanation of the symbols

[0042] 1 : Anomaly Detection System 2 : Memory Bank 3: Learning System 4: Inference System 20: Network Learning Module 22: Subsampling Module 200 : Teacher Network 210: Feature Vector Connection Layer 220 : Embedding layer 230 : Student Network

Claims

Claim 1 A Student-Teacher network-based anomaly detection system comprising: a memory bank; a learning system that trains a Student-Teacher network using normal data and stores bottleneck embedding feature vectors generated for the normal data during the training process in the memory bank; and an inference system that detects anomalies in input data using the trained Student-Teacher network and the bottleneck embedding feature vectors stored in the memory bank; wherein the learning system comprises: a network learning module that trains a Student-Teacher network using normal data and generates low-dimensional bottleneck embedding feature vectors that imply the characteristics of the normal data during the training process; and a subsampling module that subsamples the bottleneck embedding feature vectors generated for the normal data during the training process and stores the subsampled bottleneck embedding feature vectors in the memory bank. Claim 2 delete Claim 3 A Student-Teacher network-based anomaly detection system according to claim 1, wherein the network learning module comprises: a Teacher network pre-trained to extract feature vectors for input data and having fixed weights; a feature vector linking layer that combines feature vectors generated from the Teacher network; an embedding layer that is trained to condense multi-scale feature vectors of data into a low-dimensional space containing such vectors, and extracts low-dimensional embedding feature vectors from the combined feature vectors; and a Student network trained to reconstruct data from the embedding feature vectors of the embedding layer. Claim 4 A Student-Teacher network-based anomaly detection system according to claim 3, wherein the Teacher network is composed of a pre-trained encoder structure for extracting feature vectors for input data, and the Student network is composed of a decoder structure configured to learn embedding feature vectors. Claim 5 A Student-Teacher network-based anomaly detection system according to claim 1, wherein the inference system comprises an anomaly detection module, and the anomaly detection module, when new data is input, extracts an embedding feature vector through the Teacher network of a learned Student-Teacher network, calculates an outlier score by calculating the distance between the extracted embedding feature vector and the embedding feature vectors stored in a memory bank, and identifies the location of an outlier through the outlier score. Claim 6 An anomaly detection method in an anomaly detection system equipped with a memory bank and a Student-Teacher network, comprising: (a) a step of training a Student-Teacher network using normal data; (b) a step of generating a bottleneck embedding feature vector containing the characteristics of normal data during the training process of the Student-Teacher network; (c) a step of storing the bottleneck embedding feature vectors generated for normal data in a memory bank; and (d) a step of detecting anomalies in input data using the trained Student-Teacher network and the bottleneck embedding feature vectors stored in the memory bank; wherein step (c) is characterized by subsampling the bottleneck embedding feature vectors generated for normal data during the training process and storing the subsampled bottleneck embedding feature vectors in a memory bank. Claim 7 delete Claim 8 A Student-Teacher network-based anomaly detection method according to claim 6, wherein the Student-Teacher network extracts feature vectors for data through a pre-trained Teacher network, combines feature vectors generated from the Teacher network, and trains the Student network to imply multi-scale feature vectors of data in a low-dimensional space, extracts embedding feature vectors from the combined feature vectors, and trains the Student network to reconstruct data from the embedding feature vectors. Claim 9 A Student-Teacher network-based anomaly detection method according to claim 8, wherein the Teacher network is composed of a pre-trained encoder structure for extracting feature vectors for input data, and the Student network is composed of a decoder structure configured to learn embedding feature vectors. Claim 10 In claim 6, the above step (d) comprises: (d1) a step of extracting an embedding feature vector for the input data through the Teacher network of the learned Student-Teacher network when new data is input; (d2) a step of calculating an outlier score by calculating the distance between the extracted embedding feature vector and the embedding feature vectors stored in the memory bank; and (d3) a step of identifying the location of an outlier through the outlier score; a Student-Teacher network-based anomaly detection method.

Images