Strip steel anomaly detection method based on multi-scale knowledge distillation and feature information library

Abstract

The application provides a strip steel anomaly detection method based on multi-scale knowledge distillation and feature information library, and steps are as follows: an MKD-IR model is established, the model comprises a multi-scale knowledge distillation module and a block domain feature optimal storage module; the MKD-IR model is pre-trained, and intermediate features derived from a shallow network are output; the intermediate features are input into a feature storage library of the block domain feature optimal storage module, and a core subset is solved by iteration greedy approximation in NP-Hard; the MKD-IR model is optimized, and a trained model is obtained; a strip steel surface image to be detected is input into the trained MKD-IR model in step four, the maximum distance between the strip steel surface image in a data set and each nearest neighbor core subset is calculated, an anomaly score is calculated, and thus the tasks of distinguishing whether there is a defect and defect positioning are realized. The application can accurately identify unknown abnormal data, solve unknown anomaly detection problems, and effectively screen out defect samples.

Description

Technical Field

[0001] The present invention relates to the technical fields of strip steel anomaly detection and image detection, and in particular to a strip steel anomaly detection method based on multi-scale knowledge distillation and a feature information library. Technical Background

[0002] Detecting surface defects in steel strip is a critical step in the production process. Surface defects such as cracks, scale, and potholes can directly impact product quality and performance, and can even lead to production accidents. Traditional defect detection methods are slow and fail to meet the requirements for efficient inspection. This is especially true for complex defects, which rely on specialized personnel, resulting in subjective inspections and low rates of missed and false detections. Machine vision inspection technology based on deep learning has been widely applied in various fields, including road defect detection and fabric defect detection.

[0003] Although anomaly detection has become increasingly sophisticated and mature in many fields, its application in complex industrial scenarios still faces challenges. As the demand for intelligent manufacturing continues to grow, the incompleteness of industrial big data anomaly detection research and the growing demand for it are becoming a serious bottleneck hindering the advancement of intelligent industrial manufacturing. The following challenges are addressed with the existing defect detection datasets.

[0004] 1. The scarcity of anomalous data: This refers to the scarcity of accurately labeled anomalous data in industrial data. Obtaining accurate pixel-level annotation requires specialized labor and is costly. In contrast, the annotation cost of natural images is relatively low.

[0005] 2. Abnormal data imbalance: At the image level, normal industrial images far outnumber abnormal ones, due to differences in production and processing conditions. At the pixel level, the captured abnormal areas occupy a very small pixel area within the entire surface image, resulting in a very high degree of similarity between normal and abnormal samples.

[0006] 3. Diversity of anomaly representation: Industrial image anomalies often have diverse sizes, shapes, locations, and texture characteristics. From a statistical perspective, the data distribution of these anomalies is inconsistent. Therefore, it still faces some challenges in quality control and surface defect localization applications. Summary of the Invention

[0007] Purpose of the invention: The present invention proposes a strip anomaly detection method based on multi-scale knowledge distillation and feature information library, the purpose of which is to accurately identify unknown abnormal data, solve the problem of unknown anomaly detection, and effectively screen out defective samples.

[0008] The technical solutions adopted by the present invention are as follows:

[0009] The present invention proposes a strip anomaly detection method based on multi-scale knowledge distillation and feature information database, the steps of which are as follows:

[0010] Step 1: Establish the MKD-IR model. The MKD-IR model includes a multi-scale knowledge distillation module and a block domain feature optimal storage module. The multi-scale distillation module is used to accurately identify unknown abnormal data. The features and patterns learned by the source network pre-trained on ImageNet are distilled into a simpler student network. The block domain feature optimal storage module divides the feature map information of the training normal data into several regions and stores them in blocks to solve the problem of unknown anomaly detection and filter out defective samples. The MKD-IR model uses the loss function L MID and L ALL Measure when to stop training;

[0011] Step 2: Pre-train the MKD-IR model. Input the normal strip surface image from the ImageNet source network into the MKD-IR model established in Step 1 for pre-training. The normal strip surface image is input into the teacher model and student model of the multi-scale knowledge distillation module. The teacher model jointly uses the low-level features and high-level features obtained in different convolutional layers to input into the student model. The student model distills the learned features and patterns into a simpler student network and outputs the intermediate features from the shallow network.

[0012] Step 3: Input the intermediate features output in step 2 into the feature repository BANK of the block domain feature optimal storage module. The feature repository BANK uses the maximum and minimum method to obtain the core subset Solving the Core Subset via Iterative Greedy Approximation in NP-Hard

[0013] Step 4: MKD-IR model optimization, the test set containing a mixture of normal and abnormal data is input into the simple student network that has been distilled in step 2 for training, and the block domain feature score is used to explain the spatial difference. The simple student network will access the normal data information stored in all feature repositories BANK and calculate the test set to each nearest neighbor core subset BANK * The maximum distance between them is calculated, the anomaly score is calculated, and the trained MKD-IR model is obtained;

[0014] Step 5: Input the strip surface image to be detected into the simple student network of the MKD-IR model trained in step 4, and calculate the relationship between the strip surface image and each nearest neighbor core subset BANK * The maximum distance between them is used to calculate the anomaly score, thereby achieving the task of distinguishing between defects and defect location.

[0015] Furthermore, the multi-scale knowledge distillation module structure includes two branches, one is the teacher model and the other is the student model. The multi-scale knowledge distillation module aims to train a student model that can detect abnormal images in the test data. The teacher model uses the VGG-16 network and the student model uses a basic VGG network.

[0016] Furthermore, the block domain feature optimal storage module is a storage strategy that divides the feature map information of normal training data into several regions. Then, a greedy approximation algorithm is used to find the optimal sub-library, store them in blocks, and detect anomalies by comparing the differences between the test data and the features in the library.

[0017] Furthermore, the loss function L in step 1 MID and L ALL for:

[0018]

[0019]

[0020] Where, L MID It is to minimize the Euclidean distance between the activation output values ​​of each layer; L ALL is the integration of the losses of different intermediate layers; the i-th intermediate layer of the distillation learning network is defined as and When i=0, and Represents the original input, the source activation value of the intermediate layer of the teacher model is The layer activation function learned by the student model is (j) or The j-th activation value; N f for The number of neurons in the layer, N l Indicates the total number of key layers.

[0021] Furthermore, the feature repository BANK in step 3 is defined as:

[0022]

[0023] in, Where BANK is the feature repository; is a block domain feature set; is the feature map tensor; h represents the height; w represents the width.

[0024] Furthermore, the core subset in step 3 is:

[0025]

[0026] Where, represents the core set; p is the maximum value and q is the minimum value.

[0027] Furthermore, the anomaly score is solved as:

[0028]

[0029]

[0030] sorce * =||m test -m * ||2

[0031] In the formula, score is the abnormal score; score * is the maximum distance fraction; m test is an abnormal candidate; m * is the test data.

[0032] Furthermore, before step five, the surface image of the strip to be inspected is obtained, and the latest dataset SSAD-FSL is used. The dataset SSAD-FSL is input into the trained MKD-IR model obtained in step four for retraining.

[0033] Compared with the prior art, the present invention has the following advantages:

[0034] 1. This paper addresses the problem of strip anomaly detection and proposes an anomaly detection model, called MKD-IR, designed to quickly and accurately screen out anomaly images from a large number of normal strip surface images. The MKD-IR model is trained exclusively on normal data. During the testing phase, it distinguishes between normal and abnormal images by comparing test set features with features in a stored normal data database. This method utilizes a multi-scale knowledge distillation model to reduce data computational complexity and fully utilize both low-level and high-level information in normal data, enabling accurate identification of unknown anomaly data. The block-domain feature optimization storage module addresses the problem of unknown anomaly detection and screens out defective samples.

[0035] 2. The present invention adopts a block-domain feature optimal storage module, which effectively suppresses the model's tendency to extract features from natural image data, while improving the model's sensitivity to abnormal data.

[0036] 3. The method of the present invention can detect abnormal data efficiently and accurately when it is very complex and diverse, and can adapt to the situation where the amount of data is insufficient, thereby promoting the development of abnormality detection in the task of strip surface defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1Schematic diagram of the difficulties in strip steel anomaly detection of the present invention, (a) abnormal area ratio (b) various abnormal characteristics (c) normal samples;

[0038] Figure 2 MKD-IR model method flow chart of the present invention;

[0039] Figure 3 Schematic diagram of the structure of the multi-scale knowledge distillation module of the present invention;

[0040] Figure 4 A schematic diagram of the process of obtaining anomaly scores in the present invention;

[0041] Figure 5 Schematic diagram of the defect dataset of the present invention; (a) is a weld, (b) is a crescent bay, (c) is a water spot, (d) is an oil spot, (e) is a rolling pit, (f) is a foreign object, and (h) is a punching hole.

[0042] Figure 6 This is a comparative experimental graph of the present invention;

[0043] Figure 7 Schematic diagram of the simplified distillation model of the present invention and the quantitative ablation results of the same model;

[0044] Figure 8 This is a schematic diagram of the comparison results of the SSAD-FSL detection data set visualization experiment of the present invention;

[0045] Figure 9 Schematic diagram of the experimental results of samples on the SSAD-FSL detection dataset of the present invention. DETAILED DESCRIPTION

[0046] To facilitate understanding of this application, a more comprehensive description of this application will be provided below with reference to the relevant drawings. The drawings illustrate preferred embodiments of this application, but implementation of this application is not limited to the embodiments illustrated in the drawings. These embodiments are provided to facilitate understanding of the disclosure of this application.

[0047] The present invention applies an anomaly detection method based on multi-scale knowledge distillation and feature information library to strip anomaly detection, and proposes an unsupervised anomaly detection method consisting of two main components: a multi-scale knowledge distillation module (Multi-scale Knowledge Distillation Module) and a block domain feature optimal storage module (Block Domain Core Information Banks Module), to realize anomaly detection and positioning on the strip surface.

[0048] The challenges of constructing defect datasets include the following:

[0049] 1. The scarcity of anomalous data: This refers to the scarcity of accurately labeled anomalous data in industrial data. Obtaining accurate pixel-level annotation requires specialized labor and is costly. In contrast, the annotation cost of natural images is relatively low.

[0050] 2. Imbalance of abnormal data: Figure 1 This is a schematic diagram of the difficulties in detecting steel strip anomalies in the present invention. (a) shows that at the image level, normal industrial images far outnumber abnormal ones, due to differences in production and processing conditions. (b) shows that at the pixel level, the captured abnormal area occupies a very small pixel area in the entire surface image, resulting in a very high similarity between normal and abnormal samples. (c) shows a normal image, which can be used for comparison and reference.

[0051] 3. Diversity of anomaly representation: Industrial image anomalies often have diverse sizes, shapes, locations, and texture characteristics. From a statistical perspective, the data distribution of these anomalies is inconsistent. Therefore, it still faces some challenges in quality control and surface defect localization applications.

[0052] The present invention proposes a strip steel anomaly detection method based on multi-scale knowledge distillation and feature information database, which includes the following steps:

[0053] Step 1: Establish the MKD-IR model. The MKD-IR model includes a multi-scale knowledge distillation module and a block domain feature optimal storage module. The multi-scale distillation module is used to accurately identify unknown abnormal data. The features and patterns learned by the source network pre-trained on ImageNet are distilled into a simpler student network. The block domain feature optimal storage module divides the feature map information of the training normal data into several regions and stores them in blocks to solve the problem of unknown anomaly detection and filter out defective samples. The MKD-IR model uses the loss function L MID and L ALL Measure when to stop training;

[0054] Figure 2This is a flow chart of the MKD-IR model method presented in the present invention. The MKD-IR model includes a multi-scale knowledge distillation module and a block-domain feature optimization storage module. The multi-scale distillation module accurately identifies unknown abnormal data. The features and patterns learned by the source network pre-trained on ImageNet are distilled into a simpler student network. By combining low-level and high-level features obtained at different convolutional layers, the student network can deeply learn information about normal images. The simple and compact student model avoids distraction due to non-distinguishing features and enhances the ability to distinguish between normal and abnormal features. The block-domain feature optimization storage module solves the problem of unknown anomaly detection and screens out defective samples. After the student network fully transfers the features and patterns of the source program, some intermediate-level features are stored in a specific memory bank in a block-domain manner. This bank stores the core information of normal data. A greedy neighbor selection mechanism is used to find the core subset, reducing the amount of data and minimizing the impact on detection results.

[0055] like Figure 3 As shown in the figure, the multi-scale knowledge distillation module structure consists of two branches: a teacher model (TeacherModel) and a student model (StudentModel). The teacher model and student model are a knowledge distillation technique that can transfer the knowledge of a powerful but complex teacher model to a lightweight student model, thereby improving the performance and efficiency of the student model. The network structure of the teacher model and student model can have many different designs, but generally speaking, they all contain the following layers:

[0056] Input layer: receives image data;

[0057] Convolutional layer: Use convolution kernel to perform convolution operation on the input data to extract feature maps. There are multiple convolutional layers forming a convolution block, and each convolution layer is followed by a batch normalization layer and an activation layer.

[0058] Pooling layer: Use maximum pooling or average pooling to downsample the feature map to reduce parameters and computation. Usually, a pooling layer is followed by each convolution block.

[0059] Activation layer: Use activation function to perform nonlinear transformation on feature map to increase the expressive power of the model.

[0060] Residual block: Use skip connections to add input and output to achieve residual learning, avoid gradient vanishing and network degradation. There are multiple residual blocks forming a residual network, and each residual block contains two or three convolutional layers and one activation layer.

[0061] Output layer: The feature map is dimensionally transformed using a fully connected layer or global average pooling, and the prediction result is output, usually a one-dimensional vector representing the probability distribution of different categories.

[0062] The module structure aims to train a Student Model that can detect abnormal images in test data, where the Teacher Model uses a VGG-16 network, and the Student Model uses a basic VGG network. Since the training set only includes normal images without abnormal images, the intermediate knowledge of the Teacher Model on normal data is needed to guide the training of the Student Model.

[0063] To achieve this goal, the method of the invention not only uses the output features of the last layer, but also transfers the edge features of the basic layer and the semantic features of different abstraction levels of the intermediate layer to the Student Model. In this way, the Student Model can better understand the normal data, thereby improving its accuracy in detecting abnormal images. In order to save computing cost and memory usage, the invention still uses the feature distillation method for training. By using this technology, the Student Model can focus more on learning the main features and output distribution of normal images, thereby better capturing the information in the data. By compressing the source code, this technology can prevent features that cannot be distinguished between normal and abnormal data from interfering with the learning process.

[0064] The steps of feature extraction of the large teacher model are as follows: the parameters of the Teacher Model are frozen, the strip surface image with size HxWxC from the ImageNet pre-trained source network is input into the multi-scale knowledge distillation module for feature extraction, and finally the feature map size is output. At the same time, the Student Model also extracts features in the same way.

[0065] The Block Domain Core Information Banks Module is a storage strategy. The main idea of the block domain feature is to divide the feature map information of the training normal data into several regions. Then, the optimal sub-bank is found through the greedy approximation algorithm, and they are stored in blocks. Abnormalities can be detected by comparing the differences between test data and library features. Block domain feature refers to a feature extraction method that divides an image into several small blocks, then describes the features of each small block, and finally concatenates the features of each small block to form a whole feature vector. This method can utilize local region information, increasing the distinguishability and robustness of features. This paragraph describes how to store the block features in the optimal way.

[0066] The loss function of the MKD-IR model is:

[0067] The present invention defines the i-th intermediate layer of the distillation learning network as and ( and Represents the original input, the source activation value of the intermediate layer of the teacher model is The layer activation function learned by the student model is As the definition of knowledge distillation says, feature knowledge refers to the output value of the activation layer. In this method, two loss functions are defined to constrain the knowledge transfer from the teacher model to the student model. One is L between different intermediate layers. MID , which aims to minimize the Euclidean distance between the activation output values ​​of each layer; the second is at the end of the output layer of the network L ALL , integrate the losses of different intermediate layers, and the specific calculation process is shown in formulas (1)(2).

[0068]

[0069] (j) or The j-th activation value of .

[0070] At the end of the input layer, the loss functions of the above different intermediate layers are integrated. Using this loss function, L ALL Training continues to converge, which is a useful criterion for when to stop training.

[0071]

[0072] where N f for The number of neurons in the layer, N l Indicates the total number of key layers.

[0073] By jointly utilizing low-level features and high-level features obtained under different convolutional layers through a large teacher model, the student network can deeply learn the information of normal images.

[0074] Step 2: Pre-train the MKD-IR model. Input the normal strip surface image from the ImageNet source network into the MKD-IR model established in step 1 for pre-training. The normal strip surface image is input into the teacher model and student model of the multi-scale knowledge distillation module. The teacher model jointly uses the low-level features and high-level features obtained in different convolutional layers (of the teacher model) to input into the student model. The student model distills the learned features and patterns into a simpler student network and outputs the intermediate features from the shallow network.

[0075] The normal steel strip surface image is fed into the two main branches of the multi-scale knowledge distillation module, a large teacher model and a small student model. The knowledge of the source network pre-trained on ImageNet is distilled into a simpler student network. By combining low-level features and high-level features obtained at different convolutional layers, the low-level features are derived from shallow networks and are rich in spatial information with a relatively high feature resolution. The intermediate-level features are represented as x i ∈D(dataset D), j∈{1,2,3,4,5} is the feature S ψ The number of layers in the network. is a three-dimensional tensor with depth c * , height h * , with a width of w * High-level features come from deep networks and are rich in semantic information, while the feature resolution of semantic information is relatively low. The student network can deeply learn the information of normal images, avoid being distracted by non-distinguishing features, and enhance the ability to distinguish normal and abnormal features.

[0076] Step 3: Input the intermediate features in step 2 into the feature repository BANK of the block domain feature optimal storage module. The feature repository BANK uses the maximum and minimum method to obtain the core subset. Solving the Core Subset via Iterative Greedy Approximation in NP-Hard

[0077] Because distillation learning uses a network pre-trained on the ImageNet dataset, the network's training process focuses more on learning the features of abstract natural image datasets, which have very limited relevance to the strip steel industry's anomaly detection tasks and evaluation data. Consequently, this model is significantly disadvantageous for downstream strip steel surface defect classification tasks. Furthermore, unknown and variable anomaly types are often encountered during testing, further increasing the difficulty of ensuring the model's detection accuracy. To improve the model's detection accuracy, the present invention employs alternative methods.

[0078] The method of the present invention adopts a strategy to process the features in the Student Model whose accuracy is close to that of the Teacher Model, extract the core in a block-domain manner, and store it in a designated feature library. To avoid feature knowledge that is too general or biased towards natural image classification, an intermediate or mid-level feature representation is selected. This feature representation is more general, applicable to a wider range of tasks, and reduces reliance on feature knowledge of natural image classification. This strategy helps improve the generalization performance of the model and can effectively reduce the use of storage space.

[0079] After the student network fully transfers knowledge from the source program, some intermediate-level features are stored in a block-specific manner in a dedicated memory bank, which stores the core information of normal data. A greedy neighbor selection mechanism is used to find the core subset, reducing the amount of data and minimizing the impact on detection results. This memory bank is core-sampled, ensuring low inference cost and higher performance. When a test set containing a mixture of normal and abnormal data is input to the network, the network accesses all stored normal data information and evaluates the anomaly score by measuring the distance between corresponding features. This method uses block-domain feature scores to account for spatial differences and reduce the network's bias towards natural image classes.

[0080] Specifically, the selection of block domain features in the block domain feature optimal storage module is based on the student model S that completes knowledge distillation. ψ , the features at a specific network layer play an important role, and the intermediate level features are represented as x i ∈D(dataset D). In this method, j∈{1,2,3,4,5} is the feature S ψ The number of layers in the network. is a three-dimensional tensor with depth c * , height h * , with a width of w * .

[0081] use To represent the position h∈{1,…,h *},w∈{1,…,w * c at} * dimensional block-domain features. Under certain circumstances, the extraction of block-domain features is completed within a sufficiently large receptive field to capture the complex structures and patterns involving the local space, thereby improving the understanding and perception of local visual information.

[0082] Block features are crucial for stimulating local neighborhood aggregation. They can increase the size of the receptive field and improve robustness to small spatial deviations without losing the spatial resolution or usability of the feature map. To better explain this phenomenon, the previously mentioned notation is expanded. to account for uneven block sizes.

[0083] The local feature in (h,w) can be expressed as:

[0084]

[0085] Where,

[0086] f agg Is an aggregation function that aggregates the obtained feature vectors into Nearby, the method of the present invention adopts an adaptive average pooling operation. And generates h∈{1,…,h *},w∈{1,…,w *}, a representation with dimension d, which preserves the resolution of the feature map.

[0087] In summary, for a feature map tensor Its block domain feature set for:

[0088]

[0089] In order to retain the feature information used, the method of the present invention uses two intermediate feature layers j and j+1 in the Student Model. i ∈D train , aggregate each element and its corresponding block domain features together, and obtain the corresponding feature repository BANK, which is defined as:

[0090]

[0091] The core repository operates as follows:

[0092] BANK and D train The size of the training data is directly proportional to the amount of data required. The continuous increase in training data will lead to an increasingly large amount of data to be stored, and the testing and inference time will increase. Therefore, this method uses a core set sampling mechanism to reduce the amount of bank data.

[0093] The purpose of core set selection is to find a subset The solution to the problem solved on this subset is closest to the solution on the complete dataset A, especially the solution on the complete dataset can be approximated more quickly. Since different problems may have different requirements for core values, the core values ​​of interest will also vary from problem to problem. In order to ensure that the core set is selected in the block-level feature space, The coverage of the original complete data set BANK is roughly similar, and the present invention adopts the maximum and minimum method to select it.

[0094]

[0095] The exact calculation of comes from the iterative greedy approximation in NP-Hard. In order to further reduce the time of core set selection, the Johnson-Lindenstrauss theorem is used to make random linear projections To reduce the dimension d *This method has been used for a long time in the basic kNN and kMeans methods or hybrid models, by finding the nearest subset to some available set, and allowing to find an approximate solution with significantly reduced cost.

[0096] Step four: input the test set containing normal and abnormal data mixture into the simple student network distilled in step two, use the block domain feature score to explain the spatial difference, the simple student network will access the normal data information stored in the feature bank BANK, and calculate the abnormal score by calculating the maximum distance between the test set and each nearest neighbor core subset BANK * . In order to get the abnormal score, the scale w on score * is used to explain the neighborhood: if the feature of the storage bank is closest to the abnormal candidate m test , m * itself is relatively far away from the adjacent sample, so it is a rare nominal event, which will increase the abnormal score. In this way, the MKD-IR model is trained, and the bias of the network to natural images is reduced.

[0097] This step uses the block domain feature score to explain the spatial difference and reduce the bias of the network to natural image classes. The abnormal score is to find the most matching subset, and the features of the real image are matched with which subset to determine which image belongs to the damage.

[0098] Figure 4 The abnormal score flowchart of the present application is shown in the following steps:

[0099] During testing, a test data set containing normal and abnormal data mixture is given , where y = 0, y = 1 represent normal data and abnormal data respectively; the abnormal score of a complete image is obtained by integrating the abnormal matching degree of its multiple small pictures. If all the small pictures are defect pictures, the abnormal matching degree is high, and the abnormal score will be close to 1; on the contrary, if all the small pictures are normal pictures, the abnormal matching degree is low, and the abnormal score will be close to 0. Generally, 0.5 can be used as the standard for division. If it is close to 1, it must be an abnormal point, and far less than 0.5, it is not abnormal. This range will be adjusted according to the actual situation. Thus, the abnormal detection is realized.

[0100] The Mahalanobis distance is used to give the block domain of the test image position (i, j) an abnormal score, and the final abnormal score of the whole image is the maximum value of the abnormal feature BANK. The specific operation is as follows: for the feature bank BANK , and by calculating the maximum distance score score * between its feature set * and each nearest neighbor BANK testThe image-level anomaly score s∈R:

[0101]

[0102] sorce * =||m test -m * ||2

[0103] In order to obtain the abnormal score, score is used * The scale w on the feature repository is used to explain the adjacent domain: if the feature repository feature BANK is closest to the anomaly candidate m test , m * It is relatively far away from neighboring samples and is therefore already a rare nominal event, which will increase the anomaly score:

[0104]

[0105] Use m∈N b (m * ) represents the b nearest block features in BANK, used to test m * To match the original input resolution (where one might want to use intermediate network features), our method improves the result by bilinear interpolation.

[0106] Step 5: Input the strip surface image to be detected into the simple student network of the MKD-IR model trained in step 4, and calculate the relationship between the strip surface image to be detected and each nearest neighbor core subset BANK * The maximum distance between them is used to calculate the anomaly score, thereby achieving the task of distinguishing between defects and defect location.

[0107] In order to better experimentally verify the strip surface defect anomaly detection technology, the MKD-IR model can be further trained before the image to be detected is input into the MKD-IR model (before step five), and the trained MKD-IR model can then detect the image to be detected. Figure 5 This is a partial schematic diagram of the defect dataset used in this paper. The latest dataset, SSAD-FSL, is based on a publicly available dataset of surface defects in cold-rolled strip steel. It contains eight types of surface defects in cold-rolled strip steel, including punched holes, welds, meniscus, water spots, and oil spots, as well as a set of defect-free normal images. This dataset contains 3,000 grayscale images, each with a resolution of 224×224. The dataset includes both image-level and pixel-level annotations of surface defects in both cold-rolled and hot-rolled strip steel.

[0108] like Figure 5As shown in (1), the first row is the original defect image, the second row is the corresponding pixel-level label, and the third row is the image-level category label. As shown in Figure (2), it shows some defect-free strip surface images. (a) is a weld, (b) is a crescent bay, (c) is a water spot, (d) is an oil spot, (e) is a rolling pit, (f) is a foreign object, and (h) is a punching hole.

[0109] Commonly used steel strip surface defect image datasets include the steel surface defect dataset NEU-CLS, the industrial metal surface defect dataset GC10-DET, and the new benchmark dataset X-SDD for hot-rolled steel strip surface defect detection.

[0110] The dataset is named SSAD-FSL (Strip Steel Anomaly Detection for Few-Shot Learning). Each defect category includes both a training set D_Train (250 normal images) and a test set D_Test (50 abnormal images and their ground-truth images of abnormal regions, and 30-80 normal images). The resolution of each defect image is 224×224.

[0111] Example 1 Effectiveness comparison test

[0112] Figure 6 The following is a comparative experimental graph of the present invention. A large number of experiments were conducted to verify the effectiveness of the proposed method in anomaly detection. The method was first evaluated using the SSAD-FSL dataset and compared with other advanced anomaly detection algorithms, including PatchCore, DevNet, RegAD, and HTDG. To ensure experimental fairness, all experiments used the same training and testing settings and were conducted on different categories and the entire dataset. The model of the present invention was trained for 200 epochs, and the evaluation results were based on the AUROC and PRO indicators of the image. They are shown in Table 1 below:

[0113] Table 1 Quantitative results of image-level AUROC comparison experiments on the SSAD-FSL detection dataset

[0114]

[0115] Image-level AUROC is an important indicator for testing the ability of the method to identify surface defects of strip steel. The higher the value, the better the performance of the method. The results of the image-level AUROC comparison experiment are shown in the table. The proposed method has a good surface and reached a high level of 0.9868. The accuracy of the method of the present invention in processing weld defects is lower than that of other methods, which may be due to the excessive background information in the scene. Despite these limitations, the detection accuracy of the method proposed in this paper has reached 1 in identifying defects such as crescent bends, foreign matter and punching holes, which fully proves that the method has significant advantages in identifying stripe defects and structural defects.

[0116] The comparison results of ROC curve and PR curve under SSAD-FSL detection data set are as follows Figure 6 As shown in the figure, the results show that the method proposed in the present invention is superior to other methods in detecting surface defects of strip steel with tiny defects and large differences in position, size and morphology. Its PR value is higher than other advanced methods at different thresholds, which fully demonstrates the superiority and better robustness of the proposed method.

[0117] Example 1

[0118] To verify the reliability of our algorithm, we conducted the following experiments:

[0119] AUROC (Area Under the Receiver Operating Characteristic Curve) is the area under the receiver operating characteristic curve, a metric used to evaluate the performance of binary classifiers. The AUROC curve is a two-dimensional graph with the false positive rate (FPR) on the horizontal axis and the true positive rate (TPR) on the vertical axis. It reflects the performance of the classifier at different thresholds. AUROC is the area under the receiver operating characteristic (ROC) curve, and its value ranges from 0 to 1. A perfect classifier has an AUROC of 1, while a random classifier has an AUROC of 0.5.

[0120] True Positive Rate (TPR), also known as Sensitivity, refers to the proportion of samples that are correctly judged as positive among all samples that are actually positive examples. It can be expressed as:

[0121]

[0122] Among them, TP represents true positive (judged to be true, and actually true), and FN represents false negative (judged to be false, and actually true).

[0123] False Positive Rate (FPR), also known as False Alarm Rate, is the proportion of samples that are incorrectly classified as positive among all samples that are actually negative, which can be expressed as:

[0124]

[0125] where FP represents false positive (judged as true, actually false), and TN represents true negative (judged as false, actually also false).

[0126] AUROC is a more robust performance metric than accuracy, especially for imbalanced datasets where the number of samples in one class is much larger than the other. In this case, accuracy can be misleading, while AUROC can better reflect the performance of the classifier.

[0127] In the anomaly detection method of the present application, the image-level AUROC is used to measure the accuracy of the method in anomaly classification, while the pixel-level AUROC is used to measure the accuracy of the method in anomaly localization segmentation, but this pixel-level parameter is usually more tolerant to larger area defects, so PRO is used as another evaluation indicator.

[0128]

[0129] PRO (Per Region Overlap) is an index for measuring the performance of anomaly detection algorithms, which measures the overlap ratio between each real anomaly region segmentation map S g and the corresponding detected anomaly region S p . Specifically, for each real anomaly region S g , the overlap ratio between it and all detected anomaly regions S p is calculated, and the maximum value is selected as the PRO value. In order to better consider the influence of the connected part, it usually gives greater weight to the connected part containing fewer pixels, which can be expressed as:

[0130] where n are the number of connected unit regions in S p and its corresponding S g , respectively, and the number of connected unit regions it contains.

[0131] In anomaly detection, PRO can be used to evaluate an algorithm's performance in identifying anomalous regions. For example, given a dataset and an anomaly detection algorithm, PRO can be used to calculate the algorithm's detection accuracy on the dataset. PRO values ​​closer to 1 indicate higher detection accuracy, meaning the algorithm more accurately detects anomalous regions. PRO values ​​closer to 0 indicate that the algorithm may need improvement.

[0132] This paper uses two pixel-level metrics, AUROC and PRO, to quantitatively compare the results, as shown in the table. During the testing phase, both metrics evaluated the method's performance in accurately locating surface defects on steel strips, with higher values ​​indicating better performance. Defect localization can help quickly pinpoint defect locations in industrial inspections.

[0133] The proposed model performed best at the pixel-level AUROC metric, achieving 0.8924. On average for this category, the proposed method improved by approximately 2 percentage points compared to the slightly inferior PatchCore method, and by over 10 percentage points compared to other methods. While the proposed method demonstrated exceptional performance for punching defects, it struggled with more complex and variable defects such as water spot structures. This also provides guidance for further improvement, as shown in Tables 2 and 3 below:

[0134] Table 2 Quantitative results of pixel-level AUROC comparison experiment on SSAD-FSL detection dataset

[0135]

[0136] Table 3 Quantitative results of pixel-level PRO comparison experiments on the SSAD-FSL detection dataset

[0137]

[0138] The proposed method also achieved the best score for the pixel-level PRO metric, reaching a level of 1 for defects such as crescents. Compared to the AUROC metric, the PRO metric places greater emphasis on the correct classification of abnormal samples and is therefore more rigorous. Good performance on the PRO metric indicates that the proposed method can effectively identify abnormal samples with high accuracy; good performance on the AUROC metric indicates that the method has good classification capabilities across the entire dataset. This is also fully consistent with actual application scenarios, as abnormal samples are generally more important than normal samples.

[0139] Example 3 Ablation Experiment

[0140] Figure 7This figure shows the quantitative ablation results of the proposed distilled simplified model and the same model. To evaluate the contribution of each key module to the overall model performance, the proposed network was modularized, trained and tested separately, and the results were analyzed. Additionally, visualizations of the results for each model were output and analyzed.

[0141] In order to verify the effectiveness of the multi-scale knowledge distillation learning module and the block domain feature optimal storage module, the present invention adopts split training test. When verifying the effectiveness of the multi-scale knowledge distillation learning module, the complete VGG16 is used as the basic feature extraction network. In addition, for the block domain feature optimal storage module used for anomaly identification and positioning, it is replaced by the sum of the distances to the K nearest points to determine the anomaly score. The experimental results are shown in Table 4, where the corresponding modules are used in the marked representative experiments. According to the experimental results, it can be observed that, on the one hand, both modules contribute to the detection results; on the other hand, the module replacement experiment also fully illustrates that the module proposed by the present invention has certain advantages over other models. As shown in Table 4 below:

[0142] Table 4 Quantitative results of ablation experiments

[0143]

[0144] The teacher model used in the multi-scale knowledge distillation learning module is a VGG16 network pre-trained on the ImageNet dataset, while the student model uses a simplified VGG16 network. To further verify that this design method not only reduces the amount of model computation and improves the model's computational efficiency but also improves the model's detection accuracy, this section uses the same unsimplified source network for ablation experiments. The image-level AUROC values ​​are as follows: Figure 7 The results show that the simplified model has better detection results than the same model for different defect categories on the SSAD-FSL detection dataset, which fully proves the necessity of using the simplified model.

[0145] In addition, this section also conducts ablation experiments on different loss functions. These experiments use the outputs of the first three layers, the last three layers, and the final layer of the network, as well as the combined output of all intermediate layers used in this invention. Although the networks achieve good image-level AUROC results under different loss functions, the loss function used in this invention is the most competitive. This is shown in Table 5 below:

[0146] Table 5 Quantitative results of ablation experiments with different loss functions

[0147]

[0148] Example 4 Visualization Experiment

[0149] Figure 8 This is a schematic diagram of the comparison results of the visualization experiment of the SSAD-FSL detection dataset of the present invention, showing the visualization results of anomaly localization of OurNet, PatchCore, DevNet, RegAD and HTDG. The example images of the test anomaly images of each category input network are presented in the first column of the figure, and the corresponding pixel-level true value labels are in the second column of the figure. The other columns present the visualization results of different methods. By comparing these visualization results and combining them with the quantitative results in the previous section, it can be found that the method proposed in the present invention performs better in defect localization and can process the edges and structures of defects more clearly.

[0150] Example 5

[0151] Figure 9 This is a schematic diagram of the experimental results of the sample on the SSAD-FSL detection dataset of the present invention. In addition to conducting experiments on the complete dataset, the present invention also verifies the performance of the anomaly detection method in a small sample scenario. In order to test the stability of the method proposed in the present invention when processing a small amount of data, the number of training samples is changed from 2 (equivalent to 0.8% of the total training data) to 50 (equivalent to 20% of the total training data), and retraining is performed using the same module and training mechanism. The results based on the three performance indicators are as follows Figure 9 As shown in the figure, it contains three small figures: a pixel-level AUROC image, an image-level AUROC image, and a PRO image. As can be seen from the figures, the proposed method can still achieve a certain level of advanced performance even with limited data. This demonstrates the method's strong adaptability. In real-world industrial scenarios, if the production environment changes, the model only needs to be fine-tuned using a small amount of training data to meet industrial detection requirements.

[0152] It should be understood that those skilled in the art, inspired by the technical concept of the present invention, can make various improvements or changes based on the above description without departing from the content of the present invention, but this still falls within the scope of protection of the present invention.

Claims

1. A strip steel anomaly detection method based on multi-scale knowledge distillation and feature information database, characterized in that: The steps are: Step 1: Build the MKD-IR model. The MKD-IR model includes a multi-scale knowledge distillation module and a block-domain feature optimization storage module. The multi-scale distillation module is used to accurately identify unknown abnormal data. The features and patterns learned by the source network pre-trained on ImageNet are distilled into a simpler student network. The block-domain feature optimization storage module divides the feature map information of the normal training data into several regions and stores them in blocks, solving the problem of unknown anomaly detection and screening out defective samples. The MKD-IR model combines the loss functions of different intermediate layers at the end of the input layer. and Integrate to obtain ,when Stop training when the descent does not converge; Among them, the loss function and for: ； Where, It is to minimize the Euclidean distance between the activation output values ​​of each layer; is to integrate the losses of different intermediate layers; the first The middle layer is defined as and , when i=0, and represents the original input, It means that the teacher model is The j-th activation value of the layer, It means that the student model learns the teacher model in the first The j-th activation value of the layer; for The number of neurons in the layer, Indicates the total number of key layers; Step 2: Pre-train the MKD-IR model. Input the normal strip surface image from the ImageNet source network into the MKD-IR model established in Step 1 for pre-training. The normal strip surface image is input into the teacher model and student model of the multi-scale knowledge distillation module. The teacher model jointly uses the low-level features and high-level features obtained in different convolutional layers to input into the student model. The student model distills the learned features and patterns into a simpler student network and outputs the intermediate features from the shallow network. Step 3: Input the intermediate features output in step 2 into the feature storage library of the block domain feature optimal storage module , feature repository The core subset is obtained by using the minimax method , solve the core subset by iterative greedy approximation in NP-Hard ; The core subset in step 3 is: ； Where, represents the core set; p is the maximum value, q is the minimum value; Step 4: MKD-IR model optimization. The test set containing a mixture of normal and abnormal data is input into the simple student network that has been distilled in step 2 for training. The block domain feature scores are used to explain the spatial differences. The simple student network will access all feature repositories. Store normal data information and calculate the test set to each nearest neighbor core subset The maximum distance between them is used to calculate the anomaly score , get the trained MKD-IR model; Step 5: Input the strip surface image to be detected into the simple student network of the MKD-IR model trained in step 4, and calculate the relationship between the strip surface image and each nearest neighbor core subset. The maximum distance between them is used to calculate the anomaly score, thereby achieving the task of distinguishing between defects and defect location; Feature repository in step 3 Defined as: ; , It is the characteristic The number of layers in the network; For the student model based on completed distillation, is a three-dimensional tensor with a depth of , the height is , with a width of ; in, ; Where, It is a feature repository; is a block domain feature set; It is an intermediate feature tensor; Dtrain represents the training data set; is a training sample; is the block domain feature; The meaning is that the height h and width w must be integer multiples of the step length n; N is a natural number set; .

2. The strip steel anomaly detection method based on multi-scale knowledge distillation and feature information database according to claim 1 is characterized in that: The multi-scale knowledge distillation module structure consists of two branches, one is the teacher model and the other is the student model. The multi-scale knowledge distillation module aims to train a student model that can detect abnormal images in the test data. The teacher model uses the VGG-16 network and the student model uses a basic VGG network.

3. The strip steel anomaly detection method based on multi-scale knowledge distillation and feature information database according to claim 1 is characterized in that: The block domain feature optimal storage module is a storage strategy that divides the feature map information of normal training data into several regions. Then, a greedy approximation algorithm is used to find the optimal sub-library, which is stored in blocks. Anomalies are detected by comparing the differences between the test data and the features in the library.

4. The strip steel anomaly detection method based on multi-scale knowledge distillation and feature information database according to claim 1 is characterized in that: Before step five, the surface image of the strip to be inspected is obtained, and the latest dataset SSAD-FSL is used. The dataset SSAD-FSL is input into the trained MKD-IR model obtained in step four for retraining.

Images