Metal strip surface defect detection method based on the CFLOW-AD model

Abstract

Method for detecting surface defects of metal strip based on CFLOW-AD model, obtaining surface images of defect-free metal strip, constructing a training data set after amplification, building a PatchSVDD feature extraction model, importing the training data set for feature extraction pre-training to obtain a pre-trained model; passing the pre-trained model into the encoding layer of the CFLOW-AD model as a feature extractor; extracting features of the Gaussian pyramid of normal metal strip to construct a multi-scale feature pyramid; training independent decoders for each scale layer using the training data set; using the trained CFLOW-AD model to detect defective test sample images to determine the area of the defect on the image. Self-supervised learning is carried out using defect-free samples, solving the problem of difficult collection of defect samples, and solving image noise caused by imaging factors through data amplification, improving robustness and reducing noise misdetection.

Description

Technical Field

[0001] The present invention relates to a method for detecting surface defects of metal strips, specifically a method for detecting surface defects of metal strips based on the CFLOW-AD model. Background Art

[0002] Metal strips are common metal production products in the metallurgical industry and are of great significance in industrial applications. There are various types of surface defects on metal strips, with different degrees of severity, which greatly affect the service life and mechanical properties of metals. Especially for metal strips used in precision manufacturing, even minor defects will have a great impact on the final products.

[0003] Existing common methods for detecting metal strips include manual visual inspection, eddy current testing, magnetic flux leakage testing, infrared testing, and emerging machine vision technology. Among them: Manual visual inspection is overly affected by the subjective factors of inspectors and it is difficult to achieve online detection; Eddy current testing has too high costs and can only detect defects near the surface of metal strips; Magnetic flux leakage testing cannot be applied to scenarios with multiple large numbers of defects; Infrared testing has weak anti-interference ability and few detection types. Machine vision technology is currently the best solution for metal strip detection methods and equipment. However, traditional image processing technology has poor robustness in defect detection and is difficult to handle various types of defect states. Although supervised deep learning methods can adapt to surface defect detection tasks of various types and scales, they rely on the defect sample size of the dataset and are difficult to be quickly applied to defect detection scenarios. Moreover, the industrial field is currently facing problems such as unbalanced positive and negative samples, few defect samples, difficult collection, and high annotation costs, and the mainstream supervised learning methods cannot well apply to surface defect detection scenarios. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to overcome the problem that traditional image defect detection methods are limited by the defect sample size, and provide a method for detecting surface defects of metal strips based on the CFLOW-AD model.

[0005] The technical solution adopted by the present invention to solve the above technical problem is: A method for detecting surface defects of metal strips based on the CFLOW-AD model, including the following steps:

[0006] (1) Obtain the surface image of a defect-free metal strip.

[0007] (2) Augment the image obtained in step (1) to construct a training dataset.

[0008] (3) Build a PatchSVDD feature extraction model, and import the training dataset in step (2) for feature extraction pre-training to obtain a pre-trained model.

[0009] (4) Build a CFLOW-AD model, perform pooling operations on the images in the training dataset in step (2), and construct a Gaussian pyramid. Among them, the pre-trained model in step (3) is passed into the encoding layer of the CFLOW-AD model as a feature extractor to extract the features of the Gaussian pyramid of the normal metal strip, and a multi-scale feature pyramid is constructed.

[0010] (5) Use the training dataset in step (2) to train the independent decoders for each scale layer of the multi-scale feature pyramid in step (4).

[0011] (6) Use the trained CFLOW-AD model in step (5) to detect the defective test sample images and determine the area of the defect on the image.

[0012] The surface images of the defect-free metal strip in step (1) and the defective test sample images in step (6) are both grayscale images of the metal strip collected by a black and white camera.

[0013] In the aforementioned step (2), several times of amplification of the original data is completed by using rotation operations, scale transformation operations, mirror flipping operations, Gaussian blur processing, brightness transformation operations, adding random salt and pepper noise operations, and / or Mosaic operations.

[0014] In the aforementioned step (2), randomly select some images for histogram equalization processing to improve the contrast and expand the dataset samples.

[0015] In step (3), the PatchSVDD feature extraction model divides the images in the training dataset into several large image patches Patch b , and then each large image patch is divided into several small image patches Patch s , each large image patch is set with a hierarchical encoder f b , each small image patch is set with a hierarchical encoder f s , and the hierarchical encoders f of the small image patches belonging to the same large image patch s share the training parameter weights.

[0016] In step (3), in the loss function for feature extraction self-supervised learning, the Mahalanobis distance is used instead of the Euclidean distance as the loss function to evaluate the similarity of the semantic features of adjacent image patches in the feature space.

[0017] The loss function is ,

[0018] Among them, , is the image patch near, f is the encoder, and the encoding process of the image patch includes the hierarchical encoder f as described above band f s , is the feature vector calculated by the encoder;

[0019] , is a randomly sampled image patch, is another image patch within its eight-neighborhood, y is the true value of the relative position between the two image patches, , is the classifier, and the input is the feature vectors encoded from two image patches; , parameters and are used to adjust the weight ratio of samples that are difficult to predict in the task of predicting the relative position, is the probability that the predicted image patch belongs to the true relative position.

[0020] In the step (5) described above, each independent decoder is trained using the training data set to minimize the probability loss function of the decoder. Among them, the probability loss function is , is the KL divergence, is the true probability density after the feature vector is normalized to a Gaussian distribution, is the probability density estimated by the model.

[0021] In the step (6) described above, bicubic interpolation is used to upsample the multi-scale likelihood estimation result to the size of the input image to determine the area of the defect on the image.

[0022] After the defective test samples are estimated by the CFLOW-AD model, the log-likelihood of each scale obtained is converted into a probability value ,after normalization, use bicubic interpolation to upsample to the size of the input image, and aggregate all the upsampled probabilities into the anomaly score map S. The areas with high scores are the areas where the defects are located.

[0023] The beneficial effects of the present invention are as follows: Self-supervised learning is adopted using defect-free samples, which solves the problem that it is difficult to collect defective samples, and data augmentation is used to solve the image noise caused by imaging factors, improve robustness, and reduce noise misdetection. The PatchSVDD pre-trained model pre-trained by feature extraction using the defect-free sample training data set is passed into the encoding layer of the CFLOW-AD model as a feature extractor, which is more suitable for the surface defect detection of metal strips, and improves the feature extraction speed and the accuracy of defect localization.

[0024] On the basis of solving the above problems, PatchSVDD is introduced for fine-grained detection. Dividing the region blocks improves the accuracy of anomaly localization, and improving the upsampling method enhances the precision of the anomaly localization region. The loss function for PatchSVDD training is optimized, solving the problem of inconsistent feature space dimensions at multiple scales, improving the accuracy of judging the similarity of adjacent image patches, increasing the training weights of samples with difficult-to-predict relative positions, and further reducing the sensitivity to imaging noise. Description of the Drawings

[0025] Figure 1 is a flowchart of the method of the present invention.

[0026] Figure 2 is a schematic diagram of image processing for detecting a sample picture to be detected according to the present invention. Detailed Embodiments

[0027] The technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings and specific embodiments. The specific contents listed in the following embodiments are not limited to the technical features necessary to solve the technical problems recorded in the claims. At the same time, the listed embodiments are only a part of the present invention, rather than all embodiments.

[0028] The metal strip surface defect detection method of the present invention uses the surface image of a defect-free metal strip as a training sample. By obtaining the feature space distribution of normal samples, the difference in the probability density function between the image to be detected and the normal samples in the feature space is calculated to achieve anomaly localization, solving the problems of few metal strip defect samples, difficulty in covering all defect types, and large changes in defect scales, and meeting the requirements of real-time online detection of metal strips in industrial environments.

[0029] The detection method of the present invention includes the following steps:

[0030] (1) Obtain a large number of surface images of defect-free metal strips; the images are collected by a black-and-white camera, and the obtained grayscale images can adapt to various metal types and eliminate the influence of the original metal colors.

[0031] (2) Augment the images obtained in step (1) using common traditional data augmentation methods, including rotation operations, scale transformation operations, mirror flipping operations, Gaussian blur processing, brightness transformation operations, adding random salt-and-pepper noise operations, and Mosaic operations, to complete several-fold augmentation of the original data and construct a training data set.

[0032] For the images obtained by shooting or the augmented images, randomly select some images for histogram equalization processing to improve the contrast, enhance the details of the sample images, and expand the data set samples.

[0033] (3) Build a PatchSVDD feature extraction model for feature extraction pre-training. The PatchSVDD feature extraction model divides the images in the training dataset into several large image patches Patch b and then divides each large image patch into several small image patches Patch s , for example, divided into 4 small image patches Patch s to achieve anomaly localization. Each large image patch is set with a hierarchical encoder f b , and each small image patch is set with a hierarchical encoder f s . The hierarchical encoders f s of the small image patches belonging to the same large image patch s (for example, the 4 encoders f b divided above) share the training parameter weights. Each small image patch is individually encoded to generate an independent feature vector, and the encoder f

[0034] then encodes these 4 feature vectors to generate the feature vector of the large image patch. The extracted feature vector of the large image patch is used as the input for the classifier to predict the relative position between large image patches, and the loss function is calculated to achieve self-supervised learning of feature extraction.

[0035] Specifically, the PatchSVDD feature extraction model is improved as follows:

[0036]

[0037] After introducing the Mahalanobis distance, the loss function used to train the encoder for collecting semantically similar image patches and minimizing the distance between features becomes: , where is the image patch near b and f is the encoder. The encoding process of the image patch includes the hierarchical encoders f s as described above, and

[0038] , where the parameters and It is used to adjust the weight ratio of samples that are difficult to predict in the task of predicting relative positions. The training results are mainly affected by these image patches, which are set to 0.25 and 2 respectively. It is the probability that the predicted image patch belongs to the true relative position.

[0039] After introducing Focal loss for multi-class prediction of relative positions, the loss function optimized by the classifier becomes:

[0040] , where, is a randomly sampled image patch, is another image patch within its eight-neighborhood, y is the true value of the relative position between the two image patches, , is the classifier, and the input is the feature vectors obtained by encoding the two image patches. Finally, the loss function of PatchSVDD is linearly combined by the two as:

[0041] .

[0042] Import the training dataset of normal sample images obtained in step (2) into the built PatchSVDD model for training to obtain a pre-trained model.

[0043] (4), Build a CFLOW-AD model. As a generative model, it is mainly composed of an encoding layer and a decoding layer, and is used to perform pooling operations on the dataset images to construct a Gaussian pyramid. Import the pre-trained model in step (3) into the encoding layer as a feature extractor to extract the features of the Gaussian pyramid of the normal metal strip, and construct a multi-scale feature pyramid. Each scale layer obtains a feature vector as semantic information, and use a traditional position encoder to obtain a conditional vector as spatial information, and the independent decoder of each layer processes the feature vector of this layer separately.

[0044] (5), Use the training dataset of normal samples in step (2) to train the independent decoder of each scale layer of the multi-scale feature pyramid, so that the probability loss function estimated by the decoder is minimized. Among them, the probability loss function is:

[0045] , where, is the KL divergence, is the true probability density after the feature vector is normalized to a Gaussian distribution, is the probability density estimated by the model.

[0046] (6) Similarly, collect a small number of surface sample images of defective metal strips using a black-and-white camera, and use them as a test data set to evaluate the accuracy of model detection. Use the obtained defective sample test data set to test the trained model. After estimation by the CFLOW-AD model, convert the log-likelihood of each scale obtained into a probability value. , after normalization, use bicubic interpolation to upsample to the size of the input image, and aggregate all upsampled probabilities into an anomaly score map S. The areas with high scores are the defective areas, where .

[0047] Judge the accuracy of the test results. If the test accuracy meets the requirements, the above-trained samples can be used for detecting surface defects of metal strips. If the accuracy does not meet the requirements, obtain some normal sample images and continue to train the model.

[0048] Use the CFLOW-AD model trained in the above steps to detect surface defects of metal strips, judge whether there are defects on the surface of the metal strip, and determine the area of the defect on the collected image. Then, determine the position of the surface defect of the metal strip according to the area where the collected image is located on the metal strip.

[0049] The present invention combines self-supervised learning and generative models to achieve end-to-end detection of surface defects of metal strips. Use PatchSVDD for self-supervised normal sample feature learning as a pre-trained encoder to replace the original ResNet18 model pre-trained on ImageNet to improve the applicability of the model to metal strips and reduce the extraction of redundant features. And achieve fine-grained detection effects through image block operations, improving the accuracy of anomaly localization. Use Mahalanobis Distance to replace Euclidean distances as the loss function for evaluating the semantic feature similarity of adjacent image blocks in the PatchSVDD feature space, solving the problem that the scales of each feature dimension in Euclidean distance are inconsistent and correlated. Use Focal loss to replace Cross-Entropy as the classifier loss function for predicting the relative position of image blocks in PatchSVDD self-supervised learning, improving the training problem that it is difficult to classify normal sample image blocks after adding salt-and-pepper noise. Use bicubic interpolation to replace the original bilinear interpolation as the method for upsampling the probability distribution of multi-scale feature images by the CFLOW-AD decoder, sacrificing a certain operation speed but improving the edge accuracy of defect localization.

[0050] The above description of the specific embodiments is only for helping to understand the technical concept and its core idea of the present invention. Although specific preferred embodiments are used herein to describe and illustrate the technical solutions, it should not be construed as a limitation to the present invention itself. Those skilled in the art can make various changes in its form and details without departing from the technical concept of the present invention. These easily conceived changes or substitutions should all be covered within the protection scope of the present invention.

Claims

1. A method for detecting surface defects of metal strip based on the CFLOW-AD model, characterized in that: It includes the following steps: (1) Obtain the surface image of the defect-free metal strip; (2) Augment the image obtained in step (1) to construct a training dataset; (3) Build a PatchSVDD feature extraction model, and import the training dataset in step (2) for feature extraction pre-training to obtain a pre-trained model; the PatchSVDD feature extraction model divides the images in the training dataset into several large image patches Patch b , and then divides each large image patch into several small image patches Patch s , each large image patch is set with a hierarchical encoder f b , each small image patch is set with a hierarchical encoder f s , the hierarchical encoders f of the small image patches belonging to the same large image patch s share the training parameter weights, and each small image patch is independently encoded to generate an independent feature vector. The encoder f b then encodes these feature vectors to generate the feature vectors of the large image patches; In the self-supervised learning process of the PatchSVDD feature extraction model, use the Mahalanobis distance instead of the Euclidean distance as the loss function to evaluate the similarity of the semantic features of adjacent image patches in the feature space; (4) Build a CFLOW-AD model, perform pooling operations on the images in the training dataset in step (2), and construct a Gaussian pyramid. Among them, the pre-trained model in step (3) is passed into the encoding layer of the CFLOW-AD model as a feature extractor to extract the features of the Gaussian pyramid of the normal metal strip and construct a multi-scale feature pyramid. A feature vector is obtained for each scale layer. As semantic information, and use a positional encoder to obtain a conditional vector. As spatial information, the independent decoder for each layer Processes the feature vector of that layer separately. (5) Use the training dataset in step (2) to train the independent decoder for each scale layer of the multi-scale feature pyramid in step (4); (6) Use the trained CFLOW-AD model in step (5) to detect the defective test sample images. After the CFLOW-AD model estimates the defective test samples, convert the log-likelihood of each scale obtained into a probability value. , after normalization, use bicubic interpolation to upsample to the size of the input image, and aggregate all the upsampled probabilities into an anomaly score map S. The areas with high scores are the defective areas.

2. The method for detecting surface defects of metal strip based on the CFLOW-AD model according to claim 1, wherein: The surface image of the defect-free metal strip in step (1) and the defective test sample image in step (6) are both grayscale images of the metal strip collected by a black-and-white camera.

3. The surface defect detection method of metal strip based on the CFLOW-AD model according to claim 1, characterized in that: In step (2), use rotation operation, scale transformation operation, mirror flipping operation, Gaussian blur processing, brightness transformation operation, adding random salt-and-pepper noise operation, and / or Mosaic operation to complete several times of augmentation of the original data.

4. The method for detecting surface defects of metal strip based on CFLOW-AD model according to claim 1 or 3, characterized in that: In step (2), randomly select some images for histogram equalization processing to improve the contrast and expand the dataset samples.

5. The method for detecting surface defects of metal strip based on the CFLOW-AD model according to claim 1, characterized in that: The loss function is , Among them, , is an image block near, f is an encoder, is the feature vector calculated by the encoder; the represents the Mahalanobis distance between and , is a randomly sampled image patch, is another image patch within its eight - neighborhood, and y is the true value of the relative position between the two image patches, , is a classifier, and the input is the feature vectors obtained by encoding the two image patches; , parameter and are used to adjust the weight ratio of samples that are difficult to predict in the task of predicting relative positions. is the probability that the predicted image patch belongs to the true relative position.

6. The method for detecting surface defects of metal strip based on CFLOW-AD model according to claim 1, wherein: In the said step (5), each independent decoder is trained using the training data set to minimize the probability loss function estimated by the decoder, where the probability loss function is , is the KL divergence, is the true probability density after the feature vector is normalized to a Gaussian distribution, is the probability density estimated by the model.

Images